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EXECUTIVE  SUMMARY 


This  document  reports  on  the  classification  study  that  was  performed  with  the  Man-Portable 
Vector  (MPV)  sensor  at  former  Camp  Beale,  California  in  the  summer  of  201 1.  The  study  covers 
the  data  collection  survey,  data  processing  and  classification,  and  performance  analysis. 

The  MPV  technology  is  designed  to  extend  classification  of  unexploded  ordnance  (UXO)  to 
sites  where  vegetation  or  terrain  limits  access  to  vehicle-based  advanced  geophysical  platforms. 
The  former  Camp  Beale  site  presents  such  conditions  and  was  therefore  selected  to  test  portable 
UXO  detectors.  The  second-generation  MPV  prototype  is  an  electromagnetic  induction  (EMI) 
sensor  that  consolidates  on  the  same  portable  unit  a  50-centimeter  diameter  transmitter,  an  array 
of  three-dimensional  receivers  and  a  field-programmable  control  display.  The  display  provides 
immediate  visual  data  feedback  and  thus  enables  quality  assurance  and  adaptive  surveys.  The 
MPV  can  be  deployed  with  multiple  positioning  systems.  Besides  standard  GPS  technology, 
centimeter-level  accuracy  can  also  be  obtained  with  a  specific  portable  local  positioning  system 
that  is  based  on  locating  the  MPV  transmitter,  which  acts  as  a  beacon  when  turned  on.  The 
method  is  not  affected  by  natural  obstacles,  as  opposed  to  GPS  and  roving  lasers;  a  survey  can 
therefore  be  performed  in  forested  and  rugged  environments.  The  MPV  can  be  utilized  for 
detection  through  dynamic  survey  and  for  target  classification  through  static,  cued  interrogation 
survey  in  which  the  highest  quality  data  are  acquired  near  a  target.  The  first  demonstration  of  the 
current  MPV  technology  took  place  at  Yuma  Proving  Ground,  Arizona  UXO  Standardized  Test 
Site  in  October  2010.  Objectives  were  met  and  exceeded.  In  particular,  our  study  achieved  over 
90%  correct  classification  of  all  targets  within  1  meter  depth.  Beacon  positioning  accuracy  was 
confirmed.  Sensor  hardware  proved  to  be  maneuverable  and  generally  resilient.  Therefore  the 
system  was  deemed  to  be  fit  for  live-site  surveys. 

ESTCP  organized  the  first  live-site  demonstration  for  portable  systems  at  former  Camp 
Beale,  California  in  June  2011.  The  MPV  was  deployed  with  2-3  field  operators  to  acquire  cued 
interrogation  data  over  912  anomalies.  A  calibration  study  was  performed  on  the  first  day  to 
measure  the  sensor  response  over  known  targets.  A  sensor  verification  procedure  was  done  at  the 
beginning  of  each  survey  day.  Initial  survey  indicated  the  presence  of  significant  noise  from 
magnetic  soil,  and  suggested  frequent  offsets  between  flagged  target  locations  and  MPV  peak 
anomaly.  Standard  field  survey  procedures  were  adapted  to  mitigate  both  effects.  As  a  result, 
survey  productivity  was  slightly  diminished  with  an  average  of  90  targets  per  day.  The  collected 
data  were  pre-processed,  packaged  and  sent  to  several  groups  for  advanced  processing  and 
classification.  Our  classification  approach  is  detailed  in  this  report.  All  groups  were  successful  at 
efficiently  identifying  all  UXO  and  avoiding  excavation  of  over  80%  of  the  clutter,  which 
exceeded  demonstration  objectives.  The  demonstration  was  a  success. 

The  YPG  and  Beale  studies  are  the  first  stage  in  a  series  of  live  site  demonstrations  that  are 
aimed  at  establishing  the  performance,  limitations,  optimum  usage  and  costs  of  the  MPV 
technology.  The  initial  demonstrations  were  successful  and  suggested  a  strong  potential  for 
shallow  UXO  detection  and  classification.  Today  there  are  no  commercially  available  systems 
with  such  capabilities.  The  MPV  and  beacon  technology  is  scheduled  to  be  deployed  and  tested 
at  several  sites  through  year  2012  as  part  of  the  ESTCP  ongoing  live-site  demonstration  program. 
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1.0  INTRODUCTION 

This  is  the  fifth  study  in  the  series  of  Environmental  Security  Technology  Certification 
Program  (ESTCP)  demonstrations  of  classification  technologies  for  Munitions  Response  (MR). 
This  demonstration  is  designed  to  investigate  the  evolving  classification  methodology  at  a  site 
that  is  partially  wooded  with  a  mix  of  munitions  types. 

The  primary  objective  of  this  project  is  to  demonstrate  field  use  of  a  man  portable 
discrimination  system,  the  Man  Portable  Vector  (MPV)  sensor.  The  MPV  electromagnetic 
induction  (EMI)  sensor  was  designed  to  extend  advanced  discrimination  capabilities  to  sites  with 
challenging  surveying  conditions  and,  thus  allow  for  advanced  discrimination  to  be  applied  at 
most  human  trafficable  land  locations  at  moderate  cost.  The  system  is  deployed  in  conjunction 
with  a  portable  local  positioning  system  free  from  "line-of-sight"  requirements  to  facilitate 
survey  in  treed  areas,  where  traditional  location  systems  like  global  positioning  system  (GPS) 
and  laser  (e.g.,  dense  forests  and  steep  terrain)  can  fail. 

1.1  BACKGROUND 

The  Fiscal  Year  (FY)  2006  Defense  Appropriation  contains  funding  for  the  “Development  of 
Advanced,  Sophisticated  Discrimination  Technologies  for  UXO  Cleanup”  in  the  ESTCP.  As  the 
Defense  Science  Board  observed  in  2003,  “The  [...]  problem  is  that  instruments  that  can  detect 
the  buried  unexploded  ordnance  (UXO)  also  detect  numerous  scrap  metal  objects  and  other 
artifacts,  which  leads  to  an  enormous  amount  of  expensive  digging.  Typically  100  holes  may  be 
dug  before  a  real  UXO  is  unearthed!  The  Task  Force  assessment  is  that  much  of  this  wasteful 
digging  can  be  eliminated  by  the  use  of  more  advanced  technology  instruments  that  exploit 
modem  digital  processing  and  advanced  multi-mode  sensors  to  achieve  an  improved  level  of 
discrimination  of  scrap  from  UXO.” 

ESTCP  responded  by  conducting  a  Discrimination  Pilot  Study  and  funding  development  of  a 
new  generation  of  geophysical  sensors.  Results  for  the  first  three  discrimination  studies  (at  Camp 
Sibert,  Alabama,  San  Luis  Obispo,  California,  and  Camp  Butner,  North  Carolina)  were 
encouraging.  In  particular,  new  sensors  combined  with  advanced  classification  methods  allowed 
the  demonstrators  to  correctly  identify  a  significant  fraction  of  the  anomalies  as  arising  from 
non-hazardous  items  that  could  be  safely  left  in  the  ground.  Such  performance  was  facilitated  by 
favorable  survey  conditions,  vegetation,  and  modest  topographical  variations  that  permitted 
deployment  of  vehicular  and  cart-based  geophysical  platforms. 

The  results  from  these  studies  are  encouraging;  however,  there  are  many  Department  of 
Defense  (DoD)  sites  where  terrain  and  vegetation  limit  the  use  of  large  wheel-based  sensor 
systems.  Terrain  and  vegetation  conditions  (e.g.,  dense  forests  and  steep  terrain)  at  many  sites 
also  preclude  use  of  traditional  sensor  positioning  systems  like  global  positioning  system  (GPS) 
and  laser  ranging.  These  systems  can  fail  at  sites  when  terrain  and  vegetation  interfere  and  line- 
of-sight  surveying  is  not  possible. 

1.2  OBJECTIVE  OF  THE  DEMONSTRATION 

The  goal  of  this  ESTCP  demonstration  at  former  Camp  Beale  is  to  evaluate  the  MPV 
technology  for  UXO  characterization  at  a  site  where  difficult  topography  and  vegetation  preclude 
deployment  of  wheel-based  advanced  geophysical  platforms  and  traditional  "line-of-sight" 
positioning  methods  (e.g.  GPS,  laser).  Sensor  performance  is  measured  as  the  capability  to 
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correctly  classify  munitions  of  different  size  buried  at  various  depths  against  clutter.  Secondary 
metrics  such  as  reliability,  productivity,  effectiveness  of  field  procedures  and  ease  of  use  of  the 
technology  are  also  assessed. 

In  terms  of  benefits,  successful  deployment  of  the  MPV  extends  advanced  discrimination 
capabilities  to  sites  with  challenging  surveying  conditions  and  thus  allow  for  advanced 
discrimination  to  be  applied  at  most  human  trafficable  land  locations  at  moderate  cost. 


Figure  1:  Survey  with  MPV  and  beacon  in  open  field  (Beale  Combined  area). 


1.3  REGULATORY  DRIVERS 

The  Defense  Science  Board  Task  Force  on  UXO  noted  in  its  FY03  report  that  75%  of  the 
total  cost  of  a  current  clearance  is  spent  on  digging  scrap.  A  reduction  in  the  number  of  scrap 
items  dug  per  UXO  item  from  100  to  10  could  reduce  total  clearance  costs  by  as  much  as  two- 
thirds.  Thus,  discrimination  efforts  focus  on  technologies  that  can  reliably  differentiate  UXO 
from  items  that  can  be  safely  left  undisturbed. 

Discrimination  only  becomes  a  realistic  option  when  the  cost  of  identifying  items  that  may  be 
left  in  the  ground  is  less  than  the  cost  of  digging  them.  Because  discrimination  requires  detection 
as  a  precursor  step,  the  investment  in  additional  data  collection  and  analysis  must  result  in 
enough  fewer  items  dug  to  pay  back  the  investment.  Even  with  perfect  detection  performance 
and  high  Signal-to-Noise  Ratio  (SNR)  values,  successfully  sorting  the  detections  into  UXO  and 
non-hazardous  items  is  a  difficult  problem  but,  because  of  its  potential  payoff,  one  that  is  the 
focus  of  significant  current  research.  This  demonstration  represents  an  effort  to  transition  a 
promising  discrimination  technology  into  widespread  use  at  UXO-contaminated  sites  across  the 
country. 
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2.0  TECHNOLOGY 

The  MPV  technology  is  based  on  a  man-portable  EMI  sensor  with  a  transmitter  coil  and  a  set 
of  vector  receivers.  The  system  was  tested  at  Yuma  Proving  Ground  (YPG)  and  is  the  second 
generation  prototype  MPV. 

2.1  MPV  TECHNOLOGY  DESCRIPTION 
2.1.1  Electromagnetic  sensor 

The  MPV  is  a  man-portable,  wide-band,  time-domain,  EMI  sensor  composed  of  a  single 
transmitter  coil  and  an  array  of  five  receiver  units  that  measure  all  three  components  of  the  EM 
field  (Figure  2).  The  sensor  was  specifically  designed  to  (1)  be  man  portable  and  therefore  easy 
to  deploy,  maneuver  and  adapt  to  a  survey  environment,  and  (2)  acquire  data  that  is  suitable  for 
discriminating  unexploded  ordnance  (UXO)  from  non-UXO  targets.  The  MPV  sensor  head  for 
this  demonstration  comprises  a  50-centimenter  (cm)  diameter  circular  loop  transmitter  coiled 
around  a  disk  that  intermittently  illuminates  the  subsurface,  and  five  multi-component  receiver 
units  (cubes)  that  measure  the  three  orthogonal  components  of  the  transient  secondary  EM  field 
decay  with  three  air-induction  8-cm  square  coils.  One  receiver  cube  is  co-axial  with  the 
transmitters  while  four  receivers  are  placed  off-axis  around  the  transmitter  loops  in  a  cross 
pattern.  Gasperikova  et  al  (2007)  and  others  have  shown  that  having  multi-component  receivers 
placed  at  multiple  locations  can  help  reduce  the  ambiguity  between  the  size  and  depth  of  a  buried 
target  by  more  readily  allowing  recovery  of  the  components  of  the  polarizability  tensor 
associated  with  a  buried  metallic  object,  an  indicator  of  the  target  shape. 

The  MPV  is  a  programmable  instrument.  The  duration  of  the  excitation  and  time  decay 
recording  can  be  adjusted  to  any  given  time  to  accommodate  survey  needs.  The  MPV  features 
distinct  operating  modes  for  detection  and  discrimination  with  a  seamless  switch  between  the 
two.  Detection  mode  consists  of  dynamic  data  collection  for  digital  geophysical  mapping 
(DGM).  It  is  based  on  fast  EMI  transmit-receive  cycles  so  that  the  sensor  can  continuously  move 
(e.g.,  1  millisecond  [ms]  time  decay,  similar  to  Geonics  EM-61).  Discrimination  mode  is  tailored 
for  optimizing  data  quality  and  the  ensuing  target  characterization.  In  this  mode  the  sensor  is 
static  so  that  signals  can  be  stacked  (averaged  to  reduce  noise);  longer  EMI  cycles  are  applied  to 
capture  variations  in  time  decay  rates  (e.g.,  25  milliseconds  (ms),  similar  to  Geonics  EM-63). 
This  late-time  information  has  been  shown  to  be  very  useful  for  distinguishing  between  intact 
ordnance  and  thinner  walled  shrapnel  and  cultural  debris  (Billings  et  al.,  2007).  Other  currently 
available  systems  with  multiple  time  channel  measurement  capabilities  (e.g.,  Berkeley  UXO 
Discriminator  [BUD],  Geonics  EM63,  Time  Domain  EM  Towed  Array  Detection  System 
[TEMTADS])  are  required  to  be  mounted  on  a  cart  platform  due  to  the  size  and  weight  of  the 
multiple  coils  of  wire  required  for  the  transmitters  and  receivers. 

The  MPV  user  interface  has  real-time  monitoring  and  feedback  capabilities  on  data  quality, 
spatial  coverage  and  other  key  features  (signal  intensity,  time  decay,  secondary  targets,  and 
presence  of  magnetic  soil).  For  example,  the  interface  includes  a  target  location  tool  obtained  by 
displaying  the  direction  and  amplitude  of  the  measured  EMI  field  at  each  receiver  unit  (the  so- 
called  “dancing  arrows”  in  top  left  comer  of  Figure  3).  All  these  features  assist  the  field  operator 
in  efficient  data  collection,  so  that  detection  and  discrimination  data  can  be  collected  as  part  of 
the  same  survey,  thus  limiting  the  need  to  revisit  an  anomaly  for  further  characterization 
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Figure  2:  The  second  generation  MPV. 

Positioning  can  be  achieved  with  GPS  (for  open  field  survey)  or  beacon  receivers  (for  survey  in  forest  and/or 
in  steep  terrain).  Left  inset  shows  data  acquisition  (DAQ)  and  power  unit  mounted  on  a  backpack  frame. 

Right  panel  shows  view  of  sensor  head  from  above  with  cube  numbers. 

2.1.2  Geolocation 

Detection  and  classification  have  different  spatial  accuracy  requirement;  therefore  a  field 
survey  with  the  MPV  would  utilize  two  complementary  positioning  systems.  Detection  mapping 
has  decimeter  accuracy  requirements  and  can  be  performed  with  a  GPS  or  a  spool-mounted 
cotton  thread  and  optical  encoder. 

Classification  based  on  inverted  geophysical  data  requires  centimeter-level  sensor 
positioning  when  surveying  a  target  (Bell,  2005).  Because  of  the  presence  of  trees,  GPS  cannot 
guarantee  such  accuracy.  Due  to  these  limitations  we  implemented  the  MPV-beacon  positioning 
system  to  obtain  local,  accurate  sensor  positioning  when  surveying  a  target  (Lhomme  et  ah, 
2011).  The  operating  principle  consists  of  locating  the  origin  of  the  primary  field  generated  by 
the  MPV  transmitter  coil,  acting  as  a  beacon,  with  a  pair  of  EMI  receivers  rigidly  attached  to  a 
portable  beam,  placed  horizontally  on  the  ground  and  supported  by  a  pair  of  tripods  to  act  as  a 
base  station  (Figure  2).  The  azimuth  of  the  MPV  and  boom  are  recorded  with  a  digital  compass 
(3-component  attitude  sensor).  Field  trials  performed  with  the  MPV  sensor  head  showed  that 
position  and  orientation  estimated  with  this  beacon  system  were  accurate  to  within  1  cm  and  1 
degree,  respectively,  out  to  distances  of  3-4  meters  (m).  This  method  is  at  least  as  accurate  as  a 
GPS  in  open  field  when  considering  that  it  directly  predicts  the  sensor  head  location  while  the 
GPS  has  to  be  mounted  at  least  one  meter  away  from  sensor  head.  If  the  anomaly  is  marked  and 
geo-referenced  then  the  cued  interrogation  and  predicted  target  location  can  be  converted  to 
global  positions  by  placing  the  first  sounding  at  the  marked  location  and  using  the  azimuth 
measurement  to  orient  the  survey.  Alternatively,  we  can  search  a  location  where  GPS  is  accurate 
and  relate  that  location  to  the  rest  of  the  survey. 
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Figure  3:  The  MPV  detection  display  window  in  dynamic  data  collection  mode. 

The  top  left  panel  indicates  with  arrows  the  direction  of  the  nearest  compact  metallic  object  relative  to  the 
MPV  receiver  cubes  and  directs  the  operator  to  the  target  (here  the  MPV  sits  atop  the  target).  The  top  middle 
panel  shows  a  field  map  with  the  MPV  location  (red  dot)  and  azimuth  (black  line),  potential  target  locations 
(blue  dots),  and  cued  interrogation  soundings  (green  crosses). 


2.2  MPV  TECHNOLOGY  DEVELOPMENT 

Development  and  characterization  of  the  MPV  sensor  were  conducted  under  the  Strategic 
Environmental  Research  and  Development  Program  (SERDP)  MM- 1443  project  by  Engineering 
Research  and  Development  Center-Cold  Regions  Research  and  Engineering  Laboratory  (ERDC- 
CRREL)  in  Dartmouth,  New  Hampshire  (NH)  from  2005-2009  under  the  leadership  of  Benjamin 
Barrowes  and  Kevin  O’Neil.  The  first  MPV  prototype  was  built  in  2005-2006  by  David  George 
of  G&G  Sciences,  Grand  Junction,  Colorado  (CO).  It  was  tested  in  2007  at  ERDC  in  a  laboratory 
setting,  where  data  were  collected  over  a  series  of  test  ordnance  in  a  highly  controlled,  low-noise 
environment.  Interpretation  of  these  data  proved  that  the  MPV  could  meet  discrimination 
expectations  under  cooperative  survey  conditions.  The  ArcSecond  laser  positioning  system  was 
tested  in  2007  and  proved  to  deliver  accurate  location  for  local  survey.  The  ArcSecond  relied  on 
three  roaming  laser  stations  and  three  receiver  units  placed  on  top  of  the  MPV  head. 

The  SERDP  project  was  extended  in  2008  to  continue  sensor  testing  and  development  of  data 
modeling  methods.  The  first  field  tests  took  place  at  the  Sky  Research  test  plot  in  Ashland, 
Oregon  in  the  summer.  A  series  of  standard  UXO  were  surveyed  in  various  modes,  static  and 
dynamic,  while  location  was  provided  using  a  template  with  marked  locations  and  the 
ArcSecond.  Cued  interrogation  data  provided  stable  discrimination  results  and  confirmed  the 
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potential  to  extend  advanced  UXO  classification  capabilities  to  man-portable  systems  with  the 
MPV.  The  effect  of  strongly  magnetic  soil  on  EMI  sensors  were  also  investigated  during  that 
survey  as  part  of  SERDP  MM- 1573  (Len  Pasion,  Sky  Research).  It  was  found  that  the  MPV 
offered  possibilities  to  defeat  adverse  soil  effects  owing  to  its  array  structure.  These  field  trials 
also  showed  that  ArcSecond  positioning  was  impractical  and  not  reliable  enough  for  effective 
field  application  because  of  the  requirement  to  keep  all  three  rovers  in  the  field  of  view,  and  the 
long  setup  and  calibration  time.  This  experience  led  to  development  and  testing  of  an  alternative 
positioning  method  based  on  the  beacon  principle. 

The  SERDP  project  was  extended  in  2009  to  test  that  beacon  concept  and  prepare 
modification  of  the  original  MPV  prototype  for  further  field  deployment.  The  sensor  head  was 
redesigned  and  rebuilt:  lighter  materials  were  employed  and  the  circular  head  diameter  was 
reduced  to  reduce  weight  and  improve  maneuverability,  receiver  cubes  were  brought  inside  the 
transmitter  coil  to  reduce  fragility,  and  transparent  material  was  employed  to  allow  the  operator 
to  see  the  ground  through  the  unit.  Figure  4  shows  the  first  and  second  generation  sensor  heads 
for  comparison. 


Figure  4:  Two  generations  of  MPV  prototypes. 

A:  Original  sensor  with  double  transmitter,  wood  frame  and  ArcSecond  positioning  (head  weights  23  lbs). 
B:  Second  generation  sensor  and  touch-screen  control  display  (head  weights  12  lbs). 


The  MPV  was  tested  at  Yuma  Proving  Ground  (YPG)  UXO  Standardized  Test  Site  in 
October  2010.  The  Calibration  Lanes,  Blind  Grid  and  a  portion  of  the  Desert  Extreme  were 
surveyed  over  the  course  of  two  weeks.  Data  were  collected  as  part  of  a  one-pass  survey:  a 
dynamic  detection  sweep  search  would  be  performed  until  a  potential  target  would  be  detected; 
the  search  would  then  be  interrupted  and  immediately  followed  with  a  cued  interrogation  survey, 
during  which  a  series  of  static  soundings  would  be  acquired  so  as  to  map  the  spatial  extent  of  the 
anomaly;  then  the  search  would  resume.  Cued  data  were  processed  following  the  usual 
geophysical  inversion  and  model-based  feature  classification.  Stable  polarizability  transients 
were  generally  recovered  (Figure  5  for  Calibration  Lanes).  The  demonstration  was  successful 
with  detection  rate  and  probability  of  correct  classification  exceeding  90%  for  targets  buried 
within  one  meter  of  the  surface. 
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Figure  5:  Recovered  polarizabilities  for  MPV  survey  of  standard  munitions  at  YPG  Calibration  Lanes. 
Main  polarizability  (LI)  is  showed  in  red;  secondary  polarizabilities  (L2,  L3)  are  showed  in  blue  and  green. 


2.3  ADVANTAGES  AND  LIMITATIONS  OF  THE  MPV  TECHNOLOGY 

The  MPV  is  the  only  available  non-cart-based  system  that  can  acquire  multi-static,  multi- 
component  data  on  a  wide  and  programmable  range  of  time  channels.  The  MPV  offers  several 
key  benefits: 

By  being  man-portable,  the  MPV  can  be  deployed  at  sites  where  terrain  and  vegetation 
preclude  use  of  heavier,  cart-based  systems.  The  greater  portability  (no  wheels)  can  greatly 
improve  productivity,  especially  over  rough  terrain  or  for  cued  interrogation. 
Maneuverability  also  offers  the  ability  to  tilt  the  sensor  head  such  that  the  transmitter 
illuminates  the  buried  target  at  multiple  angles.  Standard  horizontal  loop  transmitters  produce 
a  strong  vertical  field  when  directly  above  the  target,  with  horizontal  field  components  being 
significant  when  the  transmitter  is  positioned  at  a  lateral  offset  from  the  target.  At  these  offset 
distances,  the  magnitude  of  the  transmit  fields  is  reduced  and  lower  signal-to-noise  ratio  data 
is  acquired.  By  tilting  the  MPV  we  can  take  multiple  “looks”  of  the  target  so  that  different 
combinations  of  the  target’s  polarization  tensor  components  are  excited,  resulting  in  more 
robust  estimates  of  the  target  parameters  and,  therefore,  more  reliable  discrimination  (Smith 
et  al.,  2005). 

For  each  measurement,  there  are  5  receivers  simultaneously  recording  three  orthogonal 
components  of  the  scattered  field  with  near-perfect  relative  positioning  among  receivers.  The 
multi-component,  multi-axis  design  relaxes  requirements  on  the  number  of  soundings 
required  to  accurately  predict  depth,  orientation,  and  target  parameters,  and  on  the  positional 
accuracy  (Grzegorczyk  et  al.,  2009).  This  number  of  soundings  is  dependent  on  the  target 
type  and  on  field  conditions.  Processing  of  low-noise  test-stand  MPV  data  with  perfect 
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positioning  has  shown  that  a  UXO  can  be  identified  with  as  few  as  5  soundings  (Barrowes  et 
al.,  2007b).  Analysis  of  MPV  data  collected  on  the  SKY  UXO  test  plot  in  Ashland,  Oregon 
over  magnetic  soil  also  show  that  a  4x4  grid  of  measurements  could  be  used  to  robustly 
recover  target  parameters. 

The  combination  of  multi-component  and  multi-time  channel  measurement  capabilities  and 
the  geometric  arrangement  of  the  receivers  offer  potential  for  identifying  and  neutralizing  the 
effect  of  magnetic  soil,  in  particular  with  soil  compensation  techniques  developed  in  SERDP 
MM- 14 14  and  1573.  When  the  MPV  is  positioned  with  its  sensor  head  parallel  to  ground 
surface,  above  magnetic  soil  over  even  ground  (in  the  absence  of  a  metallic  target),  signal  in 
the  receivers  measuring  the  radial  component  of  the  signal  (i.e.,  X  component  of  side 
receivers  and  Y  component  of  front  receiver)  should  be  equal,  and  the  horizontal  components 
of  the  co-axial  receivers  should  be  zero.  The  measured  decays  should  have  the  characteristic 
decay  of  viscous  remanent  magnetic  soil  (Figure  6).  The  effect  of  soil  can  therefore  be 
modeled  and  successful  discrimination  can  be  achieved  even  in  the  presence  of  magnetic 
soils  (Lhomme  et  al.,  2008;  Pasion  et  al.,  2008).  The  MPV  response  due  to  sensor  motion  and 
topography  over  magnetic  soil  is  predicable  (Kingdon  et  al.,  2009).  Soil  characterization  can 
also  help  exploit  data  collected  when  tilting  the  MPV  to  obtain  “multiple  looks”  at  an 
anomaly.  These  data  would  otherwise  be  difficult  to  interpret  in  presence  of  a  significant 
background  because  the  intensity  of  its  effect  would  significantly  vary  between  receivers. 
The  ability  to  accurately  model  the  MPV  signal  from  compact  metallic  targets  in  the  presence 
of  magnetic  soils  is  a  key  contributor  to  a  robust  inversion  and  discrimination  capability. 


Figure  6:  Comparison  of  MPV  response  between  magnetic  soil  and  a  metallic  target. 

Data  collected  at  Sky  Research  test  plot  in  Ashland,  OR,  where  magnetic  soils  have  shown  to  have  a 
significant  effect  on  EMI  sensors  (Pasion  et  al.,  2008).  The  recorded  signal  (left  panel)  shows  a  time 
decay  that  is  typical  of  soils  with  viscous  remanent  magnetization. 

-  The  MPV  is  fully  programmable  and  equipped  with  a  graphical  field-user  interface  that 
controls  acquisition  parameters  such  as  transmitter  waveform  characteristics,  the  duration 
of  the  excitation,  the  number  of  measurement  cycles  to  be  stacked  and  the  recording  time 
channels.  Short  acquisition  times  are  sufficient  for  detection,  whereas  discrimination 
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improves  with  stacking  many  measurements  over  a  long  time  window. 

-  The  MPV  has  highly  stable  EMI  components,  which  have  a  response  that  is  directly 

predictable  using  standard  EMI  theory.  In  field  tests  conducted  throughout  all  seasons  of 
last  year  we  verified  that  MPV  components  had  imperceptible  measurement  drift  and  were 
largely  insensitive  to  survey  conditions  (sun  exposure,  temperature).  In  general,  instrument 
drift  is  removed  by  performing  along-line  high-pass  filtering  of  the  data,  which  has  the 
potential  of  introducing  filtering  artifacts  to  the  data  and  can  bias  target  parameter 
estimates.  A  second  method  for  identifying  and  removing  instrument  drift  is  to  periodically 
measure  the  instrument  response  over  a  known  item  -  this  check  is  part  of  standard  survey 
procedures  performed  in  the  field.  The  transmitter  current  is  also  monitored  and  recorded  at 
all  times  during  the  survey  to  detect  any  variation  in  the  excitation  (e.g.,  due  to  changes  in 
battery  power). 

-  The  MPV  is  well  suited  for  small  target  discrimination.  Smaller  caliber  anomalies  have 

localized  and  rapidly-varying  spatial  response.  An  air  induction  coil  measures  a  voltage  by 
spatially  averaging  the  secondary  field  of  a  target  over  the  face  of  the  loop.  Therefore,  large 
receivers  tend  to  “smear  out”  the  secondary  field.  The  8  cm  x  8  cm  receivers  of  the  MPV 
are  typically  smaller  than  most  multi-channel  sensors  (for  example  the  Geonics  EM63 
receivers  are  50x50  cm,  TEMTADS  are  25x25  cm)  and  thus  better  suited  to  detecting  and 
sampling  the  secondary  field  over  small  targets. 
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3.0  PERFORMANCE  OBJECTIVES 


The  MPV  field  survey  at  former  Camp  Beale  serves  two  purposes:  (1)  to  characterize  the 
sensor  at  a  live  site  and  (2)  to  participate  in  ESTCP  evaluation  of  classification  methods.  The 
MPV  was  only  tested  for  the  data  collected  in  cued  interrogation  of  targets  that  have  been 
provisionally  located  using  a  Geonics  EM-61  Mkll  cart  and  marked  with  a  flag.  We  shall  refer  to 
these  anomalies  as  flagged  or  selected  targets. 

This  project  comprises  both  data  collection  and  classification  objectives.  The  first  four 
objectives,  as  described  in  Table  1,  are  mostly  intrinsic  to  the  quality  of  the  sensor  and  of  the 
deployment  method.  The  data  were  analyzed  by  the  demonstrator  and  by  other  ESTCP  partners. 
Given  that  similar  performance  was  attained  by  all  analysts,  scoring  here  only  reflects  the  results 
obtained  through  this  study  and  provides  a  realistic  assessment  of  the  instrument  quality  for 
UXO  classification. 


Table  1:  Performance  Objectives 


Performance 

Objective 

Metric 

Data  Required 

Success  Criteria 

Result 

|  Data  Collection  Objectives  | 

Detection  of  all 
munitions  of 
interest 

Percent  detected  of 
selected  anomalies 

•  Maximum  signal 
amplitude 

95%  of  targets  for 
which  depth  is  less 
than  the  minimum 
of  10  target 
diameters  and  1  m 

All  targets  were 
detected 

Repeatability  of 
instrument 
verification 
tests 

Amplitude  of  EM 
anomaly. 

Measured  target 
locations 

•  Daily  instrument 
verification  strip 

Amplitude  within 
25%. 

Location  within 

0.25  m 

Recovered 
polarizability 
within  20%. 

Target  location 
within  0.02  m 

Production  rate 

Number  of  cued 
interrogations  per 
day. 

Time  required  to 
prepare  data  for 
delivery 

•  Log  of  field  work 
and  data  pre¬ 
processing  time 
accurate  to  15 
minutes 

Survey:  100 
anomalies  per  day. 
Pre-processing 
time:  <3  minutes 
per  target 

Average  daily 
survey  rate  of  90 
targets. 

Pre-processing  time 
of  5  min  per  target 

Minimize 
number  of 
anomalies  to 
resurvey 

Number  of 
anomalies  for  which 
data  quality  can  be 
improved 

•  List  of  anomalies 
selected  for 
resurvey 

Less  than  5%  of 
anomalies  to 
resurvey 

1%  of  anomalies 
were  resurveyed 

|  Analysis  and  Classification  Objectives  || 

Maximize 

correct 

classification  of 
munitions 

Number  of  false 
targets  eliminated  at 
a  specified 
confidence  level  and 
number  of  true 
targets  kept 

•  Prioritized  dig 
list  with 
probabilities 

•  Scoring  reports 
by  IDA 

Correct 

classification  of  at 
least  95%  of 
munitions. 

Reduction  of  false 
alarms  by  >  40% 

100%  of  munitions 
were  identified. 

Lalse  alarm  was 
reduced  by  over 

80% 
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Location  and 
depth  accuracy 

Average  error  and 
standard  deviation  in 
depth,  northing  and 
easting  for  targets 

•  Ground-truth 
within  0.05  m 

•  Estimation  from 
inverted  data 

AZ  <  0.10  m 

AN  &  AE  <  0.05  m 

gZ  <  0.10  m 

gN  &  gE  <  0.15  m 

AZ  =  0.05  m 

AN  =  AE  =  0.01  m 
oZ  =  0.09  m 
aN  =  <jE  =  0.15  m 

Minimize 
number  of 
anomalies  that 
cannot  be 
analyzed 

Number  of 
anomalies  that 
cannot  be  analyzed 

•  Demonstrator 
target  parameter 

•  IDA  list  of 
anomalies 

•  Ground-truth 
labels  and  depth 

Reliable  target 
parameters  can  be 
estimated  for  > 

95%  of  selected 
targets  in  adequate 
depth  range 

Parameters  were 
inferred  for  all 
targets 

3.1  OBJECTIVE:  DETECTION  OF  ALL  MUNITIONS  OF  INTEREST 

Quality  data  should  lead  to  high  probability  of  detecting  the  targets  that  are  selected  for  cued 
interrogation  at  the  site. 

3.1.1  Metric 

The  metric  for  this  objective  was  the  percentage  of  anomalies  for  which  one  of  the  cued 
interrogation  soundings  exceeded  the  detection  threshold. 

3.1.2  Data  requirements 

Given  that  the  MPV  was  only  deployed  in  cued  interrogation  mode,  there  was  no  actual 
detection  survey.  Instead  the  amplitude  of  the  recorded  signal  could  be  expressed  in  terms  of 
Signal  to  Noise  Ratio  (SNR)  value  for  the  data  tile  that  was  associated  with  each  target. 

3.1.3  Success  criteria  evaluation  and  results 

The  objective  of  95%  detection  was  exceeded  with  a  100%  detection  rate  of  TOI  and  metallic 
clutter.  There  was  no  false  alarm;  flags  with  no  contact  generally  coincided  with  weak 
instrument  response  (low  SNR). 

3.2  OBJECTIVE:  REPEATABILITY  OF  INSTRUMENT  VERIFICATION  TESTS 

Reliability  of  survey  data  depends  on  stability  of  survey  equipment.  This  objective  concerns 
daily  verification  on  an  Instrument  Verification  Strip  (I VS)  where  metallic  targets  were  buried. 
Cued  interrogation  was  performed  to  verify  the  stability  of  the  data  amplitude  and  the  inferred 
target  parameters  and  verify  the  sensor  background  response.  Given  that  the  IVS  was  located  in 
open  field  we  could  also  directly  verify  the  accuracy  of  the  beacon  positioning  system  relative  to 
the  RTK  GPS. 

3.2.1  Metric 

The  metrics  for  this  objective  were  the  amplitude  and  decay  of  the  target  polarizability 
parameters,  the  inferred  target  location  and  the  difference  between  GPS  and  beacon  locations. 

3.2.2  Data  requirements 

The  EMI  response  for  target  soundings,  background  measurements  and  location  on  the  IVS 
were  recorded. 
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3.2.3  Success  criteria  evaluation  and  results 

This  objective  was  attained  if  the  target  parameters  amplitude  remained  within  25%  of  the 
mean  value  and  the  location  within  0.25  m.  The  objective  was  met.  We  verified  that  the 
recovered  polarizabilities  were  remarkably  stable,  that  the  inferred  location  remained  within  0.02 
m  and  that  the  beacon  remained  accurate  with  less  than  0.02  m  difference  with  GPS.  Background 
noise  was  stable  -  the  soil  response  was  negligible  at  the  IVS,  which  was  ideal  for  the  specific 
purpose  of  verifying  sensor  stability. 

3.3  OBJECTIVE:  PRODUCTION  RATE 

Ultimately  discrimination  survey  and  data  analysis  should  be  quicker  than  excavating  every 
potential  target.  This  particular  objective  only  included  the  sensor-specific  tasks  of  data 
collection  and  pre-processing.  The  latter  task  involved  preparing  data  for  distribution  to  analysts 
and  included:  digitizing  field  notes,  consolidating  all  soundings  for  each  anomaly,  inferring 
sensor  position  with  the  beacon  and  merging  with  the  EM  data,  removing  background  and 
normalizing  by  transmitter  current. 

3.3.1  Metric 

The  metrics  for  this  objective  were  the  mean  daily  survey  rate  and  the  mean  pre-processing 
time  per  anomaly. 

3.3.2  Data  requirements 

The  number  of  surveyed  anomalies  and  pre-processing  time  were  recorded  every  day. 

3.3.3  Success  criteria  evaluation  and  results 

The  productivity  objective  was  a  mean  daily  survey  rate  of  100  anomalies  and  mean  pre¬ 
processing  time  of  less  than  3  minutes  per  target.  Productivity  at  Camp  Beale  was  90  targets  per 
day  and  5  minutes  pre-processing  time  per  target.  The  lower  productivity  rate  was  due  to 
additional  tasks  that  had  to  be  performed  to  ensure  high  data  quality. 

The  original  procedure  was  based  on  a  standard  survey  pattern  of  6  points  per  target  (5  point 
in  a  square,  plus  one  tilt,  plus  one  beacon  boom  shot).  This  procedure  assumed  that  the  peak 
anomaly  was  well  located  to  mark  the  survey  center.  At  Camp  Beale  we  found  at  an  early  stage 
that  marked  target  locations  -  flags  -  often  seemed  to  be  offset  with  the  MPV  peak  location,  as 
indicated  in  the  receiver  data  collected  at  the  flag.  To  ensure  adequate  spatial  coverage  we 
changed  the  survey  pattern  to  a  3  x  3  point  grid.  Having  noticed  the  presence  of  a  significant  soil 
response,  we  also  decided  to  acquire  a  soil-background  sounding  for  each  flag  so  that  the 
potential  for  spatial  variations  in  background  signal  could  be  documented.  These  modifications 
increased  the  number  of  soundings  from  7  to  12  and  diminished  productivity. 

Pre-processing  time  was  mostly  increased  because  of  the  necessity  to  account  for  a 
significant  soil  response.  Following  data  collection,  background  soundings  had  to  be  checked 
against  a  soil  model  for  validation.  Beacon  data  processing  had  to  be  revised  to  implement  new 
verification  procedures  -  our  existing  process  was  too  simple,  having  only  been  tested  in  flat, 
horizontal  survey  environments  (test  bench  and  YPG). 


ESTCP  MR-201005:  MPV  Camp  Beale 
Demonstration  Report 


12 


March  2012 


3.4  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  TO  RESURVEY 

High  quality  data  are  necessary  to  achieve  reliable  discrimination  and  minimize  the  number 
of  anomalies  that  cannot  be  analyzed.  Insufficient  spatial  coverage  or  noisy  measurements  might 
require  that  additional  soundings  at  new  locations  or  with  increased  stacking  time  need  to  be 
collected  to  improve  the  potential  of  characterizing  a  selected  anomaly.  Data  quality  was 
reviewed  on  site  to  verify  that  each  target  had  sufficient  number  of  usable  soundings  for 
classification.  We  also  verified  that  MPV  and  beacon  receivers  were  recording  plausible  data. 

3.4.1  Metric 

The  metric  for  this  objective  was  the  number  of  targets  that  required  additional  soundings. 

3.4.2  Data  requirements 

The  number  and  identity  of  targets  that  required  resurvey  was  recorded. 

3.4.3  Success  criteria  evaluation  and  results 

The  objective  was  a  maximum  of  5%  targets  to  resurvey.  The  crew  was  sent  to  resurvey  only 
1%  (10  out  of  912  targets).  Targets  were  only  resurveyed  because  their  peak  anomaly  was 
significantly  offset  from  the  flag  location,  resulting  in  insufficient  spatial  coverage.  None  of 
these  anomalies  was  related  to  a  target  of  interest  (TOI).  Note  that  these  resurveys  could  have 
been  avoided  if  the  MPV  had  embarked  a  data  gridding  display  capability. 

3.5  OBJECTIVE:  MAXIMIZE  CORRECT  CLASSIFICATION  OF  MUNITIONS 

This  is  the  most  important  objective  of  the  classification  study:  all  shallow  munitions  should 
be  recommended  for  excavation  after  geophysical  inversion.  This  objective  required  that 
sufficient,  high-quality  data  were  collected,  and  that  the  ensuing  data  analysis,  inversion  and 
statistical  classification  process  recognized  the  presence  of  munitions. 

3.5.1  Metric 

The  metric  for  this  objective  was  the  number  of  targets  that  were  correctly  classified. 

3.5.2  Data  requirements 

A  ranked  anomaly  list  was  submitted  to  ESTCP  for  scoring  by  the  Institute  for  Defense 
Analyses  (IDA). 

3.5.3  Success  criteria  evaluation  and  results 

The  objective  of  identifying  at  least  95%  of  the  TOI  with  at  least  40%  clutter  rejection  was 
met.  All  TOI  were  found  with  over  80%  clutter  rejection. 

3.6  OBJECTIVE:  LOCATION  AND  DEPTH  ACCURACY 

Correct  target  classification  relies  on  the  capability  to  extract  accurate  target  parameters, 
including  target  location  and  depth.  The  capability  to  accurately  locate  a  target  of  interest  is  also 
of  importance  for  safe  and  efficient  site  remediation. 
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3.6.1  Metric 

The  metric  was  the  accuracy  in  estimating  target  depth  and  geographic  location. 

3.6.2  Data  requirements 

Target  location  and  depth  for  the  models  selected  for  the  submitted  dig  list  were  compared 
with  recorded  ground  truth  measurements.  Note  that  there  was  some  uncertainty  in  the 
documented  locations,  especially  in  the  densely  forested  areas. 

3.6.3  Success  criteria  evaluation  and  results 

The  objective  was  met.  The  error  between  predicted  and  observed  geographical  location  had 
a  two-dimensional  log-normal  distribution  with  mean  below  0.01  m;  the  standard  deviation  was 
0.15  m  for  all  targets  and  only  0.10  m  for  TOI.  Mean  depth  error  and  standard  deviation  were 
0.05  m  and  0.15  m  for  all  targets,  and  0.03  m  and  0.03  m  for  TOI. 

3.7  OBJECTIVE:  MINIMIZE  NUMBER  OF  ANOMALIES  THAT  CANNOT  BE 
ANALYZED 

Some  anomalies  may  not  be  classified  either  because  the  data  are  not  sufficiently  informative 
-  the  sensor  physically  cannot  provide  the  data  to  support  classification  for  a  given  target  at  a 
given  depth  -  or  because  the  data  processing  was  inadequate.  The  former  is  a  measure  of 
instrument  performance  measured  by  the  aggregate  performance  of  all  data  analysts.  The  latter  is 
a  measure  of  the  data  analysis  undertaken  in  this  study  relative  to  that  of  other  analysts. 

3.7.1  Metric 

The  metric  for  this  objective  was  the  number  of  anomalies  that  cannot  be  analyzed  by  our 
method,  and  the  intersection  of  all  anomaly  lists  among  all  analysts. 

3.7.2  Data  requirements 

Each  analyst  submitted  their  anomaly  lists.  IDA  scored  all  dig  lists. 

3.7.3  Success  criteria  evaluation  and  results 

The  objective  was  to  be  able  to  confidently  analyze  at  least  95%  of  the  selected  anomalies 
that  fall  within  the  MPV’s  detection  range.  Data  quality  was  such  that  100%  of  the  anomaly  were 
inverted  and  fitted  with  a  model.  There  were  cases  for  which  multiple  anomalies  occurred  in  the 
vicinity  of  a  flag.  Each  anomaly  was  masked  and  inverted  and  the  most  likely  TOI  for  that  given 
flag  was  determined  at  the  classification  stage.  There  were  also  cases  with  no  distinct  anomaly  at 
the  flag  location.  The  data  were  nonetheless  inverted.  In  the  case  of  the  36  No-Contact  flags,  the 
fitted  models  corresponded  either  to  small,  distant  metallic  debris  or  to  soil-like  targets  at  large 
burial  depth  (0.5- 1.2  m).  Because  such  targets  may  resemble  large  ordnance  such  as  81  mm  or 
1 05  mm,  one  of  these  targets  was  included  in  the  training  data  request. 
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4.0  SITE  DESCRIPTION 

The  site  description  material  is  extracted  from  the  Site  Inspection  Report,  which  includes  full 
details.  Camp  Beale  is  an  approximately  60,000  acre  site  located  in  Yuba  and  Nevada  Counties, 
CA.  The  demonstration  was  conducted  in  a  10  acre  area  that  is  located  within  the  historical 
bombing  Target  4  and  the  Proposed  Toss  Bomb  target  area.  An  aerial  photo  of  the  demonstration 
area  is  shown  in  Figure  7. 

4.1  SITE  SELECTION 

This  site  was  chosen  as  the  next  in  a  progression  of  increasingly  more  complex  sites  for 
demonstration  of  the  classification  process.  The  first  site  in  the  series,  former  Camp  Sibert  in 
Alabama,  had  only  one  target-of-interest  and  item  “size”  was  an  effective  discriminator.  A 
hillside  range  at  the  former  Camp  San  Luis  Obispo  in  California  was  selected  for  the  second  of 
these  demonstrations  because  of  the  wider  mix  of  munitions,  including  60-mm,  81 -mm,  and  4.2- 
in  mortars  and  2.36-in  rockets.  Three  additional  munitions  types  were  discovered  during  the 
course  of  the  demonstration.  The  third  site  chosen  was  the  former  Camp  Butner  in  North 
Carolina.  This  site  is  known  to  be  contaminated  with  items  as  small  as  37-mm  projectiles,  adding 
yet  another  layer  of  complexity  into  the  process.  The  fourth  site,  the  former  Mare  Island  Naval 
Shipyard  (MINS)  in  Vallejo,  CA,  was  selected  because  of  an  opportunity  in  the  Navy’s 
remediation  schedule  at  MINS  to  conduct  the  study  in  the  midst  of  their  ongoing  munitions 
response  project  and  prior  to  the  upcoming  removal  action  in  2012. 

This  site  was  selected  for  demonstration  because  it  is  partially  wooded  and  is  thought  to  contain 
a  wide  mixture  of  munitions.  These  two  features  increase  the  site’s  complexity  and  both 
characteristics  are  likely  to  be  encountered  on  production  sites.  The  tree  cover  poses  a  navigation 
challenge  by  increasing  the  difficulty  of  obtaining  accurate  GPS  readings.  Future  sites  including 
this  one  will  provide  additional  opportunities  to  demonstrate  the  capabilities  and  limitations  of 
the  classification  process  on  a  variety  of  site  conditions. 

4.2  SITE  HISTORY 

Prior  to  Department  of  Defense  (DOD)  usage  in  1940,  the  property  was  a  settling  point  for 
miners,  who  had  moved  on  to  other  regions  when  the  area  was  depleted  of  gold.  However,  the 
miners  that  remained  used  the  land  for  agriculture  and  cattle  grazing.  Currently,  the  former  Camp 
Beale  project  area  consists  of  multiple  land  use  property  areas.  The  east  region  is  predominantly 
undeveloped  and  used  for  cattle  grazing.  The  central  section  is  designated  as  the  Spenceville 
Wildlife  and  Recreation  Area.  Both  the  southeast  and  southwest  regions  of  the  former  Camp 
Beale  are  moderately  populated  with  rural  residential  areas.  Many  of  the  surrounding  areas  are 
used  for  ranching  activities  and  remain  undeveloped.  Detailed  information  regarding  occupation 
of  this  area  by  Native  Americans  and  early  settlers  can  be  found  within  the  ASRs. 

The  former  Camp  Beale  property  area  was  acquired  by  the  U.S.  Government  prior  to  1940 
and  consisted  of  85,654  acres.  It  was  originally  established  as  a  training  post  for  the  13th 
Armored  Division,  which  departed  in  December  1943  and  eventually  ended  up  as  part  of  the  3rd 
Army  in  Europe.  Two  other  Divisions  (the  81st  and  96th  infantries)  also  trained  at  Camp  Beale. 
The  camp  was  used  for  various  other  military  activities  such  as  a  personnel  replacement  depot, 
an  overseas  replacement  depot,  an  induction  center,  a  prisoner  of  war  encampment,  and  a  West 
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Coast  separation  center.  Camp  Beale  provided  a  complete  training  environment  for  divisional 
training,  which  included  ranges  for  all  of  the  division  weapons,  plus  areas  for  joint  training  with 
Army  Air  Corps  units  from  such  bases  as  Sacramento  and  Santa  Rosa. 

In  1943,  Camp  Beale  was  selected  as  the  site  for  the  West  Coast  Chemical  Warfare  School 
(CWS).  By  1944,  the  threat  of  chemical  warfare  with  the  Japanese  had  diminished,  and  in  June 
1944,  the  school  was  moved  to  Rocky  Mountain  Arsenal.  In  1945,  Camp  Beale  was  designated 
as  a  point  for  classification,  rehabilitation,  and  repacking  of  the  CWS  materiel  (returning  from 
the  Pacific  Theater)  until  its  closure  in  1947.  In  May  1947,  the  Camp  Beale  reservation  was 
declared  surplus  by  the  War  Department  (effective  31  May  1947)  when  it  was  placed  on  the 
surplus  list.  By  29  September  1947,  the  War  Assets  Administration  had  assumed  custody  with 
certain  regions  being  reserved  for  use  by  the  National  Guard.  It  was  during  this  time  that  a  large 
number  of  the  buildings  were  sold.  In  early  1948,  the  Air  Force  acquired  this  land  (designated  it 
as  Beale  AFB)  through  transfer  and  used  it  to  train  bombardier-navigators  in  radar  techniques. 
The  land  was  segregated  into  6  bombing  targets  (approximately  1,200  acres  each).  By  1956,  the 
Navy  began  using  two  of  the  target  areas  until  1957,  when  there  was  no  more  need  for  the 
bombing  areas  and  ranges.  At  that  time,  a  large  portion  of  the  site  (approximately  65,000  acres) 
was  declared  excess. 


Figure  7:  Aerial  photograph  of  the  former  Camp  Beale  FUDS  with  historic  ranges  overlain. 
The  original  50  acre  site  boundary  is  shown  in  the  blow-up. 


4.3  MUNITIONS  CONTAMINATION 

The  suspected  munitions  in  this  demonstration  area  include,  but  are  not  limited  to: 

•  37  mm  projectiles 

•  60mm  mortars 
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•  8 1  mm  mortars 

•  105  mm  projectiles 

At  the  particular  site  of  this  demonstration,  evidence  of  81mm  mortars  and  105mm 
projectiles  was  found  during  the  Site  Inspection  intrusive  investigation  in  2005.  It  is  also 
suspected  that  60mm  mortars  may  be  present.  In  addition,  37mm  projectiles  have  been  found 
scattered  throughout  the  former  Camp  Beale  and  are  included  as  another  suspected  munitions 
type  in  this  area.  Due  to  the  complex  historical  usage  of  this  site  over  many  years  and  the 
overlapping  network  of  historical  ranges  throughout,  it  is  also  likely  that  other  munitions  types 
beyond  those  listed  above  may  be  encountered. 

4.4  SITE  CONFIGURATION 

The  demonstration  area  totals  approximately  10  acres  divided  into  sub-areas  for  the 
MetalMapper,  the  portable  systems,  and  a  combined  area  where  both  collected  data.  The  EM61 
cart  surveyed  all  three  sub-areas  at  100%  coverage.  The  50-acre  demonstration  site  is  shown  in 
Figure  8  with  the  extent  of  the  three  sub-areas  shown.  Shape  files  delineating  the  final  sub-area 
boundaries  are  available  from  the  ESTCP  Program  Office. 
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Figure  8:  Fifty-acre  site  with  the  MetalMapper,  portable  system,  and  combined  sub-areas  delineated. 
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5.0  TEST  DESIGN 

The  goal  of  the  study  is  to  demonstrate  and  characterize  detection  and  discrimination  with  the 
MPV.  Sensor  discrimination  performance  is  characterized  as  a  function  of  the  size  and  depth  of 
the  buried  targets  and  the  presence  and  effect  of  aggravating  factors  (nearby  object,  magnetic  soil 
and  complex  terrain).  This  section  describes  data  collection  and  analysis. 

5.1  CONCEPTUAL  EXPERIMENTAL  DESIGN 

Field  data  collection  at  Camp  Beale  was  divided  into  calibration  activities  and  discrimination 
surveys.  Calibration  had  two  components.  On  the  first  day,  May  31,  201 1  we  measured  the  MPV 
response  over  a  series  of  known  targets  placed  in  different  orientations  inside  a  training  pit.  The 
dataset  was  analyzed  to  verify  the  stability  of  the  recovered  target  parameters  and  develop  a 
library  of  known  target  features.  Calibration  was  also  performed  on  a  daily  basis  by  surveying 
over  an  Instrument  Verification  Strip  (IVS),  where  5  known  targets  are  buried,  to  verify  data 
quality,  stability  of  the  target  responses  and  accuracy  of  beacon  positioning.  Details  follow  in 
Section  5.4.  The  IVS  was  also  used  after  any  repair  and  the  data  were  immediately  inverted 
before  proceeding  further. 

Discrimination  survey  activities  started  on  June  1,  2011  and  spread  over  14.5  field  days, 
including  1 .5  days  of  rain,  1 .5  days  of  repairs  and  1  rest  day.  The  survey  area  was  split  in  4  areas. 
For  each  area  a  survey  list  was  created  such  as  to  minimize  the  overall  travel  distance  to  visit 
every  anomaly.  After  the  daily  IVS  survey  field  operators  walked  to  the  next  anomaly  on  the  list 
and  proceeded  with  cued  interrogation  surveys. 

The  entire  field  deployment  was  scheduled  to  extend  over  three  weeks,  planning  for 
contingencies.  Half  a  day  was  used  at  either  end  to  unpack,  assemble  the  sensor,  set  up  the 
survey  and  base  station  and  the  converse  tasks.  Calibration  measurements  on  a  training  pit  and 
the  first  IVS  survey  took  half  a  day.  Some  data  pre-processing  and  quality  control  was  performed 
during  the  deployment,  on  and  off  site,  so  that  any  required  action  could  immediately  be  enacted. 
For  instance,  having  noticed  a  significant  soil  response  in  the  demonstration  area  and  weak 
response  in  the  calibration  area,  we  took  a  series  of  background  measurements  at  different  height 
above  ground  to  quantify  the  sensor  response  and  verify  its  predictability  (Section  5.4.3).  The 
final  pre-processing  stage  of  preparing  the  data  for  distribution  was  completed  within  two  weeks 
of  survey  termination. 

The  advanced  tasks  of  feature  extraction  by  geophysical  inversion  and  target  classification 
were  accomplished  over  the  summer  of  2011.  Some  inversions  were  initiated  while  at  Camp 
Beale  to  verify  that  meaningful  parameters  could  be  extracted;  however,  refined  strategies  for 
compensating  or  defeating  soil  background  effects  were  done  after  the  survey.  Retrospective 
analysis  and  reporting  were  scheduled  for  the  fall  and  winter. 

The  following  Gantt  chart  shows  the  schedule  for  each  phase  of  testing  and  how  the  various 
phases  are  related. 
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Table  1:  Demonstration  steps 


Preparation 

Open 

Post  survey 

Calibration 

Field 

analysis 

Tasks  and  demonstration  stages 

1.1 

1.2 

1.3 

2.1 

2.2 

3.1 

3.2 

3.3 

Mobilization  -  Demobilization 

X 

X 

CALIBRATION  ACTIVITIES:  training  pit 

X 

CALIBRATION  ACTIVITIES:  training  data 

X 

inversion  and  parameters  stability  analysis 

CALIBRATION  ACTIVITIES:  twice-daily  test  strip 

X 

X 

If  poor  performance,  change  acquisition 

X 

X 

parameters  and  resurvey 

OPEN  FIELD  SURVEY :  cued  interrogation 

X 

OPEN  FIELD  SURVEY :  data  pre-processing 

X 

X 

DATA  ANALYSIS:  feature  extraction/inversion 

X 

DATA  ANALYSIS:  classification  and  ranked  list 

X 

RETROSPECTIVE  ANALYSIS 

X 

REPORTING 

X 

5.2  SITE  PREPARATION 

Site  preparation  was  organized  by  ESTCP  Program  Office. 

5.3  SYSTEM  SPECIFICATION 

Data  were  acquired  in  cued  interrogation  mode.  The  system  was  set  for  25  milliseconds  (ms) 
excitation  and  25  ms  recording  of  EMI  transients.  This  was  accomplished  by  using  0.9  seconds 
(s)  data  blocks  with  9  repeats  (100  ms  per  cycle).  Stacking  time  was  set  to  6.3  s  by  using  7  stacks 
(effectively  9  x  7  =  63  cycles).  Digital  receivers  use  a  4  microsecond  sampling  rate.  The  data 
were  recorded  with  133  logarithm-spaced  time  gates  (0.05%  gate  width)  from  0-25  ms. 

Local  positioning  was  achieved  with  the  beacon  system.  An  Applied  Physics  543  orientation 
sensor  was  mounted  at  the  far  end  of  the  MPV  handle  to  measure  azimuth.  The  three-axis  sensor 
data  were  also  used  for  verifying  the  pitch  and  roll  inferred  from  the  beacon  measurements. 
Practically,  the  beacon  boom  was  laid  on  the  ground  within  2  meters  of  the  survey  flag.  Boom 
orientation  was  recorded  by  placing  the  MPV  head  at  the  boom  center  and  lining  up  the  MPV’s 
main  direction  (Y-axis)  with  the  boom’s  direction.  To  facilitate  quality  checks,  the  boom  was 
generally  oriented  in  the  North-South  direction,  and  because  the  terrain  was  sloped,  the  boom 
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was  placed  uphill  from  the  flag.  After  data  processing,  beacon-derived  positions  were  located 
relative  to  the  local  flag  and  the  geographic  North,  and  subsequently  globally-referenced  using 
the  supplied  GPS  coordinates  of  each  flag.  A  Novatel  real-time  kinematic  (RTK)  GPS  was  also 
mounted  on  the  MPV  handle.  It  was  used  to  locate  pre-programmed  flag  locations  and  verify 
beacon  accuracy  whenever  enough  satellites  were  visible,  in  particular  at  the  IVS  and  in  the 
open-field  “combined  area”,  where  we  started  the  demonstration.  The  GPS  operated  at  20  Hertz; 
its  base  station  was  located  at  the  NE  marked  monument,  which  position  was  convenient  to 
service  the  entire  site. 

The  original  sounding  pattern  consisted  in  placing  the  MPV  head  at  the  four  corners  and 
center  of  an  imaginary  square  centered  on  the  anomaly.  Having  noticed  that  the  flag  location  did 
not  always  coincide  with  the  MPV  peak  anomaly,  we  modified  the  survey  pattern  to  a  9-point 
grid.  This  approach  had  several  benefits:  first,  setting  a  Standard  Operating  Procedure  (SOP) 
relieved  the  operator  from  having  to  constantly  monitor  the  sensor  display  to  guess  the  target 
location;  second,  this  reduced  operator-dependency  on  data  quality;  finally,  additional  soundings 
can  be  useful  to  assess  and  mitigate  background  effects.  In  addition  to  the  survey  pattern,  one 
sounding  was  acquired  by  tilting  the  head  back  by  45  degrees  above  the  center  or  the  detected 
anomaly  peak.  If  the  operator  found  that  peripheral  soundings  showed  significant  signal  above 
background,  then  additional  soundings  were  collected  as  an  attempt  to  capture  the  full  spatial 
decay  of  the  flagged  target. 

5.4  CALIBRATION  ACTIVITIES 

Calibration  is  necessary  for  verifying  proper  sensor  operation  and  calibrating  sensor  response 
over  known  targets. 

5.4.1  Training  pit 

Classification  is  easier  when  the  typical  response  of  expected  targets  is  known  -  these 
responses  can  be  aggregated  into  a  library  for  comparison.  The  demonstration  site  was  expected 
to  host  37,  60  and  81  mm  mortars  and  105  mm  projectiles,  4”-long  steel  cylinder  (ISO)  and 
potentially  other  types  of  UXO.  A  sample  of  these  known  targets  was  supplied  by  ESTCP  to 
measure  their  response  in  a  test  pit.  The  target  set  was  augmented  with  metallic  fragments. 


Figure  9:  Calibration  targets  used  in  training  pit. 

The  top  row  includes,  starting  from  the  left,  60  mm  mortar  body  (no  nose  and  fins),  37  mm,  ISO  cylinder, 
empty  57  mm  casing,  folded  scrap  and  81  mm.  The  center  piece  is  a  fragment  of  a  72  mm  caliber  projectile, 

and  the  bottom  item  is  a  105  mm. 
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Each  sample  was  successively  placed  inside  a  clutter-free  training  pit  and  surveyed  in  cued 
interrogation  mode.  Measurements  were  taken  with  the  target  oriented  at  a  vertical  and  oblique 
angle,  nose  up  and  down,  and  in  horizontal  position.  Depth  and  orientation  were  recorded 
(Figure  10).  The  data  were  subsequently  inverted  to  verify  that  stable,  target-specific  features 
could  be  obtained.  These  features  were  included  into  the  target  library  for  classification. 


Figure  10:  Cued  interrogation  training  at  test  pit. 


A:  Cued  interrogation  on  a  flat  surface,  following  standard  survey  procedure.  The  pit  remained  hollow,  no 
dirt  was  put  back  in  to  fill  it  in;  therefore  a  sheet  of  plywood  was  placed  to  cover  the  hole. 

B:  Depth  measurement  of  a  60  mm  mortar  body. 


5.4.2  IVS 

Control  on  sensor  drift  was  performed  through  daily  survey  over  the  IVS,  where  known 
targets  were  buried  in  a  clutter-free  environment.  The  targets  were  an  ISO,  37  mm,  60  mm  and 
81  mm  mortars  and  105  mm  projectile.  Each  target  location  was  marked  with  a  flag;  no  detection 
survey  was  required.  A  cued  interrogation  survey  was  performed  for  each  target,  following  a  9- 
point  grid  plus  tilt.  Data  were  generally  inverted  soon  after  the  survey  to  verify  data  quality, 
stability  of  the  recovered  parameters  and  accuracy  of  the  beacon  positioning  system.  The  IVS 
calibration  survey  generally  took  20  minutes. 

5.4.3  Background  measurements 

Soil  had  a  distinct  red  tint  in  the  demonstration  area  that  suggested  presence  of  iron  oxides. 
These  oxides  are  known  to  have  an  effect  on  EMI  sensors  (Pasion  et  al.,  2008;  Lhomme  et  al., 
2008).  Because  oxidation  depends  on  soil  iron  content  and  degree  of  weathering,  there  can  be 
significant  spatial  variations  at  a  site  with  varied  topography.  To  document  these  potential 
variations  and  control  response  due  to  magnetically  active  soil,  we  collected  a  background  shot 
for  each  target  or  cluster  of  targets.  The  measurements  were  made  by  placing  the  sensor  head  on 
the  ground,  at  least  2  m  away  from  the  nearest  flag,  and  quantitatively  verifying  the  typical 
amplitude  and  decay  response  of  magnetic  soils  (Figure  1 1). 

Background  responses  at  the  IVS  and  in  the  field  are  shown  in  Figure  12  along  with  a 
schematic  that  illustrates  the  expected  magnetic  soil  response  for  the  MPV.  The  IVS  shows  a 
weak  background  response.  In  contrast,  field  soundings  showed  the  characteristic  viscous 
remanent  magnetization  that  affects  specific  receivers  (vertical  and  radial)  with  log-linear  time 
decay  (soil  decays  in  1/t).  Observation  of  this  rule  was  quantitatively  verified  during  the  pre- 
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processing  stage  to  confirm  validity  of  the  clean  soil  background  measurements.  In-air 
measurements  away  from  any  other  potential  noise  sources  were  also  periodically  acquired.  Use 
of  these  background  measurements  and  noise  compensation  methodologies  are  discussed  in  a 
later  section. 


Figure  11:  Background  measurement  in  open  field  area,  where  red  soil  indicates  presence  of  iron  oxides. 
The  sensor  head  is  placed  on  the  ground,  away  from  the  surrounding  flags.  The  operator  examines  the 
recorded  response  to  confirm  the  absence  of  metal  objects  in  the  sensor’s  field  of  view. 


Cube  #1  Cube  #2  Cube  #3  Cube  #4  Cube  #5 


10?  10°  10J  10"  10?  10°  io:  10°  10?  10° 

Time  (ms)  Time  (ms)  Time  (ms)  Time  (ms)  Time  (ms) 


Figure  12:  Effect  of  magnetic  soil  on  the  MPV  sensor. 

Left:  Background  measurements  at  IVS  and  in  field  with  MPV  sensor  head  placed  on  ground  away  from  flag. 
Right:  Schematic  of  magnetically  active  soil  response.  The  response  mirrors  primary  field  emitted  by  MPV 
circular  transmitter.  Therefore  it  affects  X-component  (East-West)  data  measured  on  cubes  2  and  4,  Y- 
component  (North-South)  data  of  cubes  1  and  5,  and  Z-component  (vertical)  data  for  all  cubes,  as  shown  in 
left  panel. 
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5.5  DATA  COLLECTION 
5.5.1  Scale 


Data  were  collected  over  912  targets  spread  in  4  different  areas.  Their  spatial  distribution  is 
shown  in  Figure  13.  The  GPS  base  station  and  IVS  were  located  in  the  central  valley;  the  survey 
areas  were  distributed  on  the  side  slopes.  Representative  pictures  of  each  survey  area  are  shown 
in  Figure  14. 


Figure  13:  Sounding  locations  in  Camp  Beale  demonstration. 

The  training  pit  was  located  next  to  the  IVS,  on  the  central  valley  floor. 
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Figure  14:  Photographs  of  the  four  survey  areas  (chronologically  ordered). 

A:  Open  field,  sloped,  combined  area;  the  base  camp  was  installed  on  the  valley  floor;  the  GPS  base  station  is 
located  above  a  rocky  outcrop  near  a  tire  track  in  the  top  left  corner  of  the  picture.  B:  The  steep,  densely 
treed  area  A.  C:  The  gently  rolling,  sparsely  treed  area  C.  D:  The  rolling  and  treed  area  B. 

5.5.2  Survey  pattern  and  sample  density 

Cued  interrogation  soundings  were  collected  around  the  marked  target  location  (ground  paint 
and  flag).  The  first  sounding  was  acquired  at  the  marker.  The  subsequent  soundings  were 
generally  acquired  in  a  virtual  3x3  points  grid  pattern  as  in  Figure  15.  Recorded  data  were 
displayed  immediately  after  each  sounding  acquisition  to  help  the  operator  estimate  the  anomaly 
location  and  assess  spatial  coverage.  This  was  accomplished  by  examining  recorded  decay 
curves  (Figure  16)  and  verifying  that  the  furthest  receivers  were  measuring  weak  signals 
(background).  If  residual  target  signal  was  detected  on  peripheral  receivers  then  additional 
soundings  were  collected  to  improve  spatial  coverage.  For  instance,  if  the  MPV  front  receivers 
showed  above-background  signal  when  the  MPV  was  placed  in  position  2  (Figure  15,  left  panel), 
then  a  sounding  would  be  collected  30  cm  to  the  right  of  position  2.  If  a  nearby,  non-flagged 
interfering  target  was  detected,  then  supplementary  soundings  were  acquired  to  help  separate  the 
two  sources.  Finally,  the  MPV  sensor  head  was  placed  above  the  anomaly  peak  at  45  degree 
pitch  to  stimulate  a  different  excitation  direction  with  high  signal  while  remaining  as  close  as 
possible  to  the  target.  Only  one  tilt  test  was  performed  because  it  was  practically  difficult  to  keep 
the  sensor  steady  on  its  side. 
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Figure  15:  Soundings  pattern  for  cued  interrogation  with  MPV. 

Left:  Nine-point  survey  pattern  is  centered  on  target.  Right:  Sensor  head  is  tilted  with  45 
degree  pitch  for  transverse  excitation  at  short  standoff. 


We  had  originally  planned  to  adopt  a  5 -point  square-like  pattern  with  the  possibility  for  the 
operator  to  acquire  additional  data  points  if  the  recorded  signal  indicated  a  large  footprint.  This 
process  would  have  required  intense  focus  from  the  operator,  who  would  be  expected  to  interpret 
decay  curves  for  each  receiver  to  locate  the  target  and  verify  that  signal  had  vanished  on  the 
peripheral  soundings.  Practically,  this  process  put  too  high  a  demand  on  the  field  operator, 
especially  when  considering  that  1000  soundings  were  collected  on  a  daily  basis.  The  method 
was  also  devised  on  the  assumption  that  the  first  sounding  was  located  at  the  signal  peak. 
However,  we  found  on  the  first  surveyed  targets  that  the  flag  location  generally  did  not  coincide 
with  the  target  location,  as  suggested  by  the  MPV  receiver  cubes  (Figure  17).  For  these 
anomalies  the  peak  signal  suggested  that  the  target  could  be  located  15-30  cm  off  to  the  side.  To 
relieve  pressure  on  the  operator  and  maximize  our  chances  of  obtaining  full  anomaly  coverage, 
the  standard  operating  procedure  was  modified  to  a  9-point  grid  pattern  with  40-50  cm  spacing. 
Given  that  receiver  cubes  were  rigidly  mounted  with  18.5  cm  separation,  the  maximum  signal 
sampling  space  was  approximately  20  cm. 


Figure  16:  Typical  target  response  above  a  buried  metallic  target. 

The  Z-component  data  show  that  the  target  is  closest  to  center  cube  (#3)  and  equally  distant  from  lateral 
cubes  2  and  4,  while  signal  in  5  seems  to  be  due  to  background.  The  Y  data  confirm  that  target  is  buried 
between  front  and  back  cubes  (1,  5)  and  X  data  confirm  that  target  is  located  between  side  cubes  2  and  4. 
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Figure  17:  Gridded  image  of  the  vertical  component  receiver  data  for  the  first  four  targets  collected  at  Beale. 
The  flag  location,  indicated  with  the  red  star  at  coordinates  (0,  0),  is  generally  offset  from  the  signal  peak. 
This  observation  motivated  use  of  a  conservative  3x3-point-grid  survey  pattern. 


5.5.3  Quality  checks 

Data  quality  checks  were  performed  during  and  after  the  survey.  As  noted  above  each 
sounding  was  immediately  examined  to  verify  spatial  coverage  and  to  ensure  that  all  receivers 
were  properly  operating.  Any  acknowledged  abnormal  sounding  was  re-acquired  and  tagged  in 
the  field  notes  to  differentiate  it  from  acceptable  soundings.  If  a  receiver  failed  then  the  survey 
was  interrupted  to  verify  that  the  failure  was  persistent  and  consider  solutions.  David  George, 
who  built  the  sensor,  was  invited  to  participate  in  the  survey  and  was  present  on  site  for  parts  of 
the  study  with  tools  and  spares  for  repair.  Data  quality  was  also  controlled  by  daily  visual  data 
review.  Any  problem  would  be  investigated  and  the  affected  anomalies  would  be  resurveyed.  For 
instance,  we  found  that  for  10  out  of  912  targets  the  spatial  coverage  was  insufficient.  This  was 
due  either  to  a  large  target  offset  (up  to  50  cm,  e.g.,  Figure  18),  or  a  shrunk  survey  imprint 
(operator  distraction),  or  absence  of  a  target  at  a  flag  (target  841,  which  flag  was  displaced  by  3 
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m  -  we  returned  and  found  paint  on  the  ground  at  the  reported  GPS  location,  brought  the  flag 
back  and  resurveyed). 


Figure  18:  Example  of  target  with  large  spatial  offset  requiring  resurvey  to  obtain  sufficient  spatial  coverage. 
The  original  data  is  showed  on  the  left  panel  and  the  recollect  on  the  right.  The  original  data  were  roughly 
inverted  and  indicated  a  large,  deep  target  with  low  confidence.  Instead  of  potentially  recommending  the 
target  for  excavation,  data  were  recollected;  analysis  indicated  a  small  piece  of  metallic  clutter. 

Beacon-positioning  was  controlled  through  different  steps.  Firstly,  field  operators  took  care 
of  placing  the  beacon  boom  approximately  1.5-2  m  away  from  the  target  flag,  and  keep  the  data 
acquisition  system  (DAQ)  and  power  unit  at  least  2  m  away  from  the  beacon  boom  cubes  to 
avoid  distorting  the  local  EM  field.  Secondly,  beacon  signal  were  monitored  on  the  sensor 
display  on  graphs  similar  to  those  of  Figure  16  along  with  the  other  receivers.  Weak  beacon 
amplitude  could  be  detected  by  the  operator.  Finally,  data  were  pre-processed  and  visualized 
onsite,  when  possible,  to  infer  beacon  locations,  compare  with  GPS  and  verify  accuracy  (Figure 
19). 

We  also  ensured  that  all  anomalies  were  visited  by  pre-programming  their  GPS  coordinates 
and  displaying  their  location  on  the  sensor  display  map  (Figure  3).  This  tool  was  useful  for 
approaching  new  flags.  Each  visited  target  was  automatically  crossed  on  the  map  to  confirm 
survey.  Given  that  the  tree  canopy  was  not  completely  opaque,  the  GPS  was  always  functioning, 
though  sometimes  at  low  accuracy,  so  that  we  could  obtain  a  rough  estimate  of  our  current 
location  relative  to  flagged  targets. 

5.5.4  Data  handling 

Data  were  stored  as  .tern  files  on  the  DAQ  and  converted  to  .csv  files  before  every  battery 
change.  A  copy  of  all  .tern  and  .csv  files  was  kept  on  the  DAQ,  on  a  portable  hard-disk  drive  and 
on  the  field  laptops  that  were  used  for  reviewing  the  data.  Field  notes  recorded  the  association  of 
target  identities  and  file  numbers,  in  addition  to  any  relevant  comment  for  non-standard  events. 
Notes  were  digitized  every  day  by  taking  pictures  of  the  notes  and  filling  out  a  spreadsheet  to  be 
used  for  pre-processing. 
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Sounding  number 


Figure  19:  Sounding  locations  for  a  cued  interrogation. 

This  type  of  figures  was  used  for  quality  control  over  beacon  accuracy  and  spatial  coverage.  The  survey 
pattern  followed  a  3x3  points  grid.  The  left  panel  shows  a  3-dimensional  representation  of  the  target 
locations,  as  independently  predicted  by  the  beacon  and  the  GPS  -  locations  coincide  within  better  than  2  cm. 
Stars  mark  receiver-cube  locations  and  illustrate  the  spatial  sampling.  Large  circles  outline  the  MPV 
transmitter.  The  10th  sounding  is  the  tilted  test,  highlighted  with  thicker  lines.  The  top  right  panel  shows  the 
sounding  locations  relative  to  the  beacon  boom  (green  line  with  cubes  number  6  and  7)  and  a  gridded  image 
of  the  relative  elevation  of  each  receiver  cube  -  terrain  is  gently  sloping  away  from  the  beacon  boom.  The 
bottom  right  shows  the  predicted  elevation  for  the  GPS  (red)  and  beacon  (black  cross  for  center  cube,  blue 
dots  for  side  cubes);  given  that  the  beacon  is  a  local  prediction,  we  can  compare  the  relative  accuracy  of  the 
beacon  and  GPS  by  solving  for  the  rotation  and  translation  that  matches  their  locations  (least-squares 
minimization). 


5.5.5  Data  summary 

The  raw  data  reside  with  SKY  and  G&G  Sciences  and  include  one  configuration  file  and  one 
EM  data  file  for  each  sounding.  There  are  approximately  10,000  files  for  the  entire 
demonstration.  Files  were  merged  and  pre-processed  as  described  in  Section  6.1. 

5.6  VALIDATION 

Target  validation  was  organized  by  ESTCP  Program  Office. 
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6.0  DATA  ANALYSIS  AND  PRODUCTS 

6.1  PREPROCESSING 

6.1.1  Field  data 

The  MPV  DAQ  computer  records  data  streams  from  the  sensor  head,  beacon  receivers, 
attitude  sensor  and  GPS  with  the  GPS  time  stamp.  The  DAQ  saves  the  data  into  a  .tern  binary 
fde.  Each  sounding  data  is  converted  to  a  .csv  fde  without  any  data  alteration.  Several  pre¬ 
processing  stages  are  performed  before  delivery  to  the  analysts. 

6.1.2  Positioning  data 

The  beacon  receiver  data,  the  MPV  transmitter  current  and  the  digital  compass  reading  are 
combined  to  infer  the  MPV  head  location.  The  method  is  similar  to  a  dipole  inversion  (Lhomme 
et  al.,  2011)  and  generally  yields  better  than  2  cm  accuracy.  When  the  GPS  has  maximum 
accuracy  (Q  factor  is  equal  to  4),  its  reading  and  the  attitude  sensor  data  can  be  utilized  to  predict 
the  sensor  head  location  for  comparison  with  the  beacon  prediction.  In  case  of  aberration  with  a 
beacon  location,  GPS  readings  and  field  notes  can  provide  information  to  diagnose  the  issue. 
Otherwise  the  sounding  may  be  discarded.  In  practice  there  was  no  detected  issue  with  the 
beacon  at  Beale. 


6.1.3  Delivered  data 

The  pre-processed  data  were  delivered  in  individual  files,  each  containing  all  the  data 
associated  with  one  target:  EMI  data,  beacon-inferred  sensor-head  location,  GPS-derived  head 
location  and  nearby  background  sounding.  The  file  name  indicated  the  target  label.  Data  files  for 
the  field  data  collection  were  supplied  to  ESTCP  Program  office  for  distribution,  whereas 
calibration  files  from  the  IVS  were  directly  sent  to  parties  involved  with  the  MPV  analysis. 

The  data  were  delivered  in  two  formats.  The  first  dataset  was  fully  pre-processed.  The  data 
were  normalized  by  the  transmitter  current  to  remove  EMI  data  dependency  on  battery  power, 
which  can  vary  by  up  to  20%.  In-air  background  was  subtracted  from  the  EMI  data  to  remove  the 
transmitter  ringing  that  can  still  be  detected  at  early  time  (0. 1-0.3  ms).  The  data  were  reduced  to 
32  time  gates  by  logarithmic  integration  (through  the  same  process  that  had  reduced  the  4 
microsecond  signal  to  the  recorded  133  time  gates).  A  nearby  background  soil  signal,  clean  of 
any  metallic  clutter,  was  associated  to  each  target  for  the  analyst  to  devise  his  own  soil 
correction.  The  second  dataset  was  raw.  Transmitter  current,  related  in-air  measurement,  nearby 
soil-background  measurement  and  all  133  time-gated  data  were  supplied  to  the  analyst. 

One  complicating  factor  was  the  occurrence  of  close  targets.  There  were  many  targets  with 
less  than  one  meter  spacing,  sometimes  30  cm.  Under  such  circumstances  the  soundings 
collected  between  the  two  neighboring  targets  may  receive  responses  from  both  targets. 
Characterization  of  each  target  may  therefore  be  subjected  to  the  capability  to  separate  the 
overlapping  target  responses.  Doing  so  requires  accurate  relative  positioning  of  the  data  tiles 
obtained  for  each  target  so  that  the  joint  dataset  can  be  analyzed.  Similarly,  we  observed  cases 
where  a  detected  anomaly  would  be  located  between  two  flags  (Figure  20).  There  too  the  two 
tiles  have  to  be  stitched  together  for  analysis,  which  requires  accurate  positioning  between  tiles. 
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Figure  20:  Ambiguous  target  identity  located  between  two  survey  flags. 
Gridded  image  of  the  vertical  component  EM  data. 


Figure  21:  Survey  of  multiple  targets  with  same  beacon  station  and  flag-GPS  positional  ambiguity. 

Field  flag  locations  where  well  placed  relative  to  the  buried  targets,  though  their  placement  significantly 
differed  with  reported  GPS  coordinates.  Analysis  was  further  complicated  with  presence  of  secondary 
targets.  Targets  proximity  required  masking  for  inversion  of  individual  anomalies  (green  circle). 


The  survey  protocol  was  devised  with  these  constraints  in  mind.  Whenever  a  cluster  of  flags 
was  encountered,  all  targets  were  surveyed  using  the  same  reference  beacon  boom  station  so  that 
targets  could  be  accurately  located  relative  to  one  another.  This  was  particularly  important  in  the 
woods,  where  GPS  accuracy  could  deteriorate  and  lead  to  misleading  interpretation  of  the 
distance  between  flags  (Figure  21).  Relying  on  the  reported  GPS  locations  to  assemble 
independently-collected  neighboring  tiles  could  lead  to  gross  errors.  In  total  we  surveyed  352 
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flags  in  a  multi-target  setting,  in  which  several  targets  were  deemed  to  be  close  enough  to  be 
covered  with  the  same  beacon  station  so  that  time  could  be  saved  by  not  having  to  relocate  the 
beacon  station,  and  overlapping  signatures  could  be  addressed  if  warranted.  Out  of  these  352, 
there  were  62  triplets  and  8  quadruplets. 

The  delivered  data  were  organized  so  that  the  analyst  could  recognize  and  handle  the 
occurrence  of  multiple  close  targets.  For  each  target,  the  provided  file  started  with  the  data  block 
pertaining  to  that  target,  followed  with  data  blocks  for  its  neighboring  targets  if  their  field-flag 
location  was  located  less  than  1  m  away.  We  assumed  that  further  targets  would  not  interfere. 

6.2  TARGET  SELECTION  FOR  DETECTION 

The  detection  survey  was  based  on  Geonics  EM-61  data.  Targets  were  selected  by  Parsons 
and  ESTCP.  Target  names  and  locations  were  supplied  for  the  cued  interrogations  presented  in 
this  study. 

6.3  PARAMETER  ESTIMATION 

The  pre-processed  data  presented  in  Section  6.1  were  imported  in  the  MatLab-based 
UXOLab  software  jointly  developed  by  Sky  Research  and  the  University  of  British  Columbia 
and  used  in  numerous  ESTCP  and  SERDP  projects.  Each  anomaly  was  treated  separately.  In 
general  there  were  approximately  10  soundings  per  anomaly. 

The  data  were  inverted  using  a  three-dipole  instantaneous  polarization  model.  Inversion  setup 
parameters  such  as  noise  estimation  were  decided  upon  examination  of  the  training  pit  data  and 
field  background  measurements.  Solutions  with  one  or  multiple  targets  were  investigated  for 
every  selected  target.  Decisions  regarding  the  number  of  targets  at  a  given  location  were  left  to 
be  made  through  statistical  classification  by  prioritizing  munitions-like  models.  Inversion  results 
were  reviewed  by  an  experienced  geophysicist  to  identify  any  potential  issues  with  the  inversion 
setup  or  the  sensor  data. 

6.3.1  Feature  extraction  with  dipole  model 

In  the  EMI  method,  a  time  varying  field  illuminates  a  buried,  conductive  target.  This 
illuminating  -  or  primary  -  field  induces  currents  in  the  target  that  subsequently  decay, 
generating  a  decaying  secondary  field  that  is  measured  at  the  surface.  These  data  are  then  used 
to  estimate  the  position,  orientation,  and  parameters  related  to  the  target’s  material  properties  and 
shape.  In  the  UXO  community,  it  is  commonly  assumed  that  the  secondary  field  can  be 
accurately  approximated  as  a  point  dipole  (e.g.,  Bell  et  al.  2001,  Pasion  and  Oldenburg  2001, 
Gasperikova  et  al.  2009).  The  process  of  estimating  the  parameters  of  the  dipole  forward  model 
from  the  data  is  called  data  inversion. 

The  dipole  model  assumes  that  the  time-varying  secondary  magnetic  field  B(t),  is  due  to  a  point 
dipole  m(t)  located  at  r: 

BW  =  J\4)|3ff-1).  (!) 

4  7JT 

where  f  =  r/|r|  is  the  unit- vector  pointing  from  the  dipole  to  the  observation  point,  I  is  the  3  x  3 
identity  matrix,  /u0  =  4 7rx  10"7  H/m  is  the  permittivity  of  free  space  and  r  =  |r|  is  the  distance 
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between  the  center  of  the  object  and  the  observation  point.  The  dipole  induced  by  the  interaction 
of  the  primary  field  Bc  and  the  buried  target  is  given  by: 

m(0  =  — M(0-Bo,  (2) 

Vo 


where  M(7)  is  the  target’s  polarizability  tensor.  The  tensor  reflects  the  characteristics  of  the 
buried  target  in  terms  of  its  shape,  size,  and  material  properties.  It  is  written  as: 


Lx{t)  0  0 

0  L2(t)  0 

.  0  0  L3{t) 


(3) 


The  magnitude  and  decay  of  the  polarizability  tensor  elements  is  a  function  of  the  size  and 
electromagnetic  properties  of  the  target.  The  relative  sizes  of  the  tensor  elements  are  indicative 
of  the  target  shape.  If  ordering  tensor  elements  such  that  Lj>L2>Ls,  then  a  steel  body-of- 
revolution  (BOR)  would  have  for  a  rod- like  object  and  L]=L 2  for  a  plate-like  object. 


The  target  parameters  of  the  dipole  model  (i.e.  location,  orientation,  and  polarizability  tensor 
elements)  are  estimated  in  the  process  of  data  inversion.  The  objective  of  data  inversion  is  to 
determine  the  set  of  parameters  that  most  accurately  predict  the  observed  data.  Numerical 
optimization  methods  are  used  to  determine  the  parameters  that  minimize  a  data  misfit  objective 
function: 

minimize  ^(m)  =  —  ||  V~ll2{d°^s  -  .F(m/arge())  ||  subject  to  mmf  <  m(arge(  <  /nsup  ,  (4) 

where  Vd  is  the  data  covariance  matrix,  which  includes  a  user-defined  noise  estimate,  d°bs  is  the 
observed  data,  F  is  the  forward  modelling  operator,  mtarget  is  the  recovered  vector  of  target 
parameters,  and  minf  and  msup  are  the  inferior  and  superior  bounds  on  model  parameters.  The 
noise  estimate  corresponds  to  not  explicitly  modelled  phenomena  that  can  affect  the  data,  such  as 
sensor  noise,  geologic  effects  and  positional  error.  We  assume  that  noise  is  composed  of  a  base 
level  plus  a  factor  that  is  proportional  to  the  signal  amplitude.  In  UXOLab  noise  parameters  are 
set  by  the  user  before  an  inversion.  The  dipole  model  is  an  ideal  candidate  for  inversion  because 
(1)  the  forward  model  is  physics  based  with  parameters  that  are  indicative  of  target 
characteristics,  and  (2)  the  forward  model  is  fast  to  compute,  which  is  important  since  numerical 
optimization  implies  an  extensive  search  of  parameter  space  that  involves  numerous  calculations 
of  the  forward  model. 


6.3.2  Data  quality 

The  main  factors  affecting  data  quality  are  instrument  noise,  environmental  noise  and 
positional  error.  The  first  two  mostly  depend  on  the  sensor  characteristics  and  on  the  EM 
interference  with  the  background  host,  namely  the  magnetic  soil  found  at  Beale.  The  baseline 
noise  was  subtracted  from  sensor  data  before  inversion;  the  variability  of  that  noise  was  utilized 
to  define  the  base-level  noise  estimate  of  the  covariance  matrix. 

The  most  accessible  mean  for  estimating  instrument  noise  is  to  perform  in-air  measurements 
away  from  any  interference  source,  assuming  that  there  is  negligible  atmospheric  EM 
disturbance  (in  practice,  radio  air-waves  are  eliminated  owing  to  the  fast  4  microsecond  sampling 
rate  and  stacking).  These  measurements  were  performed  several  times  a  day  to  monitor  any 
potential  drift.  The  recorded  signals  were  normalized  by  transmitter  current  intensity  and 
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analyzed  (Figure  22).  The  early  time  spikes  correspond  to  ringing  after  the  transmitter  shuts  off; 
receivers  saturate  as  indicated  by  the  truncated  spikes.  Each  receiver  has  an  individual  response 
that  depends  on  the  degree  of  coupling  with  the  vertical  transmitter  -  vertical  component 
receivers  show  the  largest  coupling  with  lasting  effect  until  0.3  ms  -  and  on  their  location 
relative  to  one  another  and  to  the  transmitter  switch  -  the  switch  is  located  between  cubes  #4-5, 
which  explains  their  particular  X-Y-component  responses.  Signals  were  highly  repeatable  and 
rarely  deviated  from  the  median  signal;  the  standard  deviation  at  early  time  is  one  order  of 
magnitude  smaller  than  the  median.  The  sensor  was  therefore  remarkably  stable.  Stability  of 
each  receiver’s  response  allowed  subtraction  of  that  signature  from  the  target  data.  In  practice  we 
used  the  data  in  the  0.1-25  ms  range.  Residual  sensor  signature  has  limited  repercussions  for 
classification  because  it  has  fast  decay  -  most  of  the  signal  has  vanished  by  0.3  ms.  For 
comparison,  target  response  for  the  deepest  37  mm  and  81  mm  ordnance  at  Beale  exceeded  10 
milli-Volt  (mV)  on  all  receivers  and  had  a  slow  decay,  thus  dominating  the  sensor’s  signature 
past  0. 1  ms.  Soil  background  response  is  more  significant  and  discussed  in  the  next  section. 


Figure  22:  Stability  of  in-air  background  measurements  for  each  MPV  receiver. 

The  absolute  value  of  the  recorded  signals  in  showed  on  a  log-log  graph.  Downward  spikes  correspond  to 
crossovers.  Individual  measurements  are  shown  with  black  lines.  Their  median  is  indicated  in  red  and  the 
standard  deviation  in  yellow  -  there  is  negligible  variability  in  early  time  ring-down  effects. 

The  Y-component  of  the  central  receiver  cube  had  temporary  problems.  Given  that  there 
remained  sufficient  data  for  classification  with  14  operational  receivers  and  that  the  electrical 
engineer  in  charge  of  sensor  maintenance  was  momentarily  away  from  the  site,  we  proceeded 
with  the  field  data  collection.  This  issue  only  concerned  one  day’s  worth  of  data  collection  and 
did  not  significantly  affect  classification  results  (a  similar  dysfunction  had  happened  at  the  YPG 
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demonstration;  we  had  verified  stability  of  the  recovered  target  parameters  when  omitting  one 
central  receiver).  A  cable  connection  issue  was  subsequently  diagnosed  and  fixed. 

Positional  accuracy  can  also  be  considered  as  a  data  quality  factor  for  a  spatially  varying 
signal.  Positional  errors  are  equivalent  to  measurement  errors  with  the  amplitude  of  the  target 
response.  Therefore  we  parameterized  data  errors  as  a  floor  value  plus  a  percentage  of  the  data 
amplitude  to  factor  in  positional  errors.  There  is  no  direct  way  of  verifying  accuracy  of  the  MPV 
positioning  system  in  the  field.  We  compared  predicted  sensor  location  with  beacon  and  GPS, 
where  applicable,  and  generally  found  1-2  cm  relative  difference  between  the  two  (Figure  23). 


Relative  positional  error  (cm) 

Figure  23:  Relative  accuracy  of  beacon  positioning  system  compared  with  GPS. 

The  beacon  method  predicts  the  MPV  transmitter  location.  The  GPS  and  attitude  sensor  are  mounted  at  the 
far  end  of  the  MPV  handling  boom  and  can  also  be  used  to  predict  the  MPV  transmitter/receiver  head 
location  with  2  cm  accuracy  in  open  field  (1.5  m  lever  arm).  Relative  positional  error  was  compared  in  open 
field  in  the  combined  area  and  at  the  I  VS 


6.3.3  Background  noise  from  magnetic  soil 

As  seen  in  a  previous  section,  magnetically  active  soil  has  a  typical  signature  with  MPV  data: 
vertical  and  radial  component  data  have  higher  amplitude  and  log-linear  decay.  We  devised  a  test 
to  screen  background  soundings  and  verify  that  the  response  was  clean  of  any  metallic  clutter 
and  that  the  sensor  head  was  flat  on  the  ground.  The  test  consisted  of  computing  a  linear  fit  to  the 
decays,  verifying  the  slope  and  residual  error  of  that  fit  and  verifying  that  the  vertical  component 
data  had  similar  amplitude  on  the  four  lateral  cubes.  Background  soundings  that  met  these 
criteria  were  accepted  as  representative  backgrounds  for  correcting  the  target  data  and  support 
the  following  analyses. 

Having  noticed  a  distinct  soil  response  during  the  field  survey,  we  opted  to  collect  a 
background  sounding  for  every  target  to  hedge  against  large  variations.  The  soil  response  maps 
of  Figure  24  show  significant  spatial  variability  in  each  survey  area,  which  justifies  our  field 
strategy.  The  largest  variability  was  observed  in  the  densely  treed  area  (panel  #3).  One  distinct 
patch  shows  a  4-fold  increase  in  background  amplitude  above  baseline.  Variability  can  also  be 
observed  in  the  amplitude  and  decay  signal  shown  for  each  MPV  receiver  in  Figure  25,  where 
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typical  soil  responses  are  confirmed.  The  time-varying  standard  deviations  for  each  receiver 
were  derived  from  these  curves  and  used  for  parameterizing  data  error  expectations. 

The  densely  treed  area  had  a  distinct  surface  cover:  the  top  of  the  hill  (left  side)  had  long, 
thick  grass  and  the  lower  part  was  bare.  The  former  prevented  the  sensor  to  be  in  contact  with  the 
ground  whereas  the  latter  allowed  the  sensor  to  directly  rest  on  dirt.  Soil  response  approximately 
follows  that  spatial  pattern.  This  could  suggest  that  some  of  the  variability  might  be  due  to 
ground  clearance.  In  a  later  section  (7.1)  we  present  sensor  height  tests  where  we  quantified  that 
effect;  our  analysis  shows  that  the  sensor  should  be  lifted  by  15-20  cm  to  achieve  such  an  effect, 
which  is  much  greater  than  the  grass  cushion.  The  treed  area  clearly  contained  a  sector  with 
elevated  magnetic  soil  content. 
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Figure  24:  Spatial  variability  map  for  the  magnetic  soil  background  response. 

Each  panel  corresponds  to  a  survey  area;  panel  position  mimics  geographical  distribution  (l=combined  area, 
3=densely  treed,  valley  runs  vertically  between  1-3  and  2-4).  The  signal  amplitude  is  the  sum  of  all  receivers 
and  time  gates  between  0.2-0.57  ms.  Black  crosses  indicate  verified  soil  background  sounding  locations. 
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Figure  25:  Soil  background  response  in  densely  treed  area.  Typical  magnetic  soil  responses  are  observed. 
Large  amplitude  signal  in  radial  directions  (X  component  of  cubes  2  and  4  and  Y  component  of  cubes  1  and 
5)  and  in  vertical  direction  (Z  component  on  all  cubes),  and  log-linear  decay  after  turnoff  spike  has  vanished. 

6.3.4  Strategies  for  defeating  magnetic  soil 

Several  methods  were  attempted  to  compensate  for  magnetic  soil  disturbance.  The  first 
method  consisted  of  background  subtraction.  For  each  interrogated  target,  we  took  the  ratio  of 
the  validated  local  background  amplitude  and  that  of  the  target  sounding  with  the  weakest 
response.  If  soil  was  uniform  then  the  latter  would  have  larger  or  equal  amplitude  and  the  ratio 
would  be  smaller  than  unity.  To  account  for  spatial  variability  we  allowed  a  50%  tolerance  on 
that  ratio  and  used  the  resulting  factor  to  define  the  amount  of  background  to  subtract  to  all  target 
data.  The  corrected  data  were  subsequently  inverted  for  the  presence  of  one  or  two  dipole 
sources  (method  described  in  following  section).  In  the  case  of  two-dipole  inversions  applied  to 
unique  targets,  the  secondary  dipole  model  would  generally  fit  the  residual  soil  signal  that 
remained  after  the  correction. 

The  second  method  was  based  on  not  correcting  for  background  soil  and  letting  the  dipole 
model  handle  soil  effects.  This  method  consisted  of  a  two  stage  approach.  First  we  applied  a 
single  dipole  inversion  while  using  only  the  soil  insensitive  components.  As  seen  in  Figure  25, 
the  X-component  data  of  cubes  1,  3  and  5  and  the  Y-component  data  of  cubes  2,  3  and  4 
remained  unaffected  by  soil  when  the  sensor  was  parallel  to  the  ground.  Given  9  soundings  per 
target,  there  could  be  sufficient  informative  soundings  to  solve  for  a  single  buried  target.  Then,  at 
the  second  stage,  we  used  that  solution  to  constrain  location  of  one  target  as  part  of  a  two-dipole 
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source  inversion  with  all  the  sensor  data,  so  that  the  second  dipole  might  account  for  soil  or  other 
disturbances.  A  similar  method  was  successfully  applied  by  Pasion  on  highly  magnetic 
Cambodian  soil  (SERDP  MR- 1573). 

All  validated  models  were  then  injected  in  the  classifier.  In  general,  both  methods  returned 
similar  models  because  target  signal  was  larger  than  soil  disturbance.  Retrospective  analysis 
showed  similar  performance  except  for  large  composite  targets  that  require  two  dipoles  to  model 
heterogeneity. 

6.3.5  Overlapping  signatures 

At  highly  contaminated  sites,  anomalies  often  overlap  due  to  the  close  proximity  of  multiple 
targets.  Our  data  inversion  strategy  for  overlapping  targets  was  to  decompose  the  inverse 
problem  into  several  steps.  Each  step  resolved  a  subset  of  model  parameters.  The  first  step  was 
to  solve  for  non-linear  location  parameters  and  subsequently  solve  for  linear  polarization 
parameters.  With  an  optimal  estimate  of  locations  and  dipolar  polarizations,  the  orientations  and 
polarizability  decays  of  each  object  could  be  extracted  from  estimated  magnetic  polarizability 
tensors.  Our  multi-object  inversion  approach  has  been  successfully  tested  as  part  of  previous 
Discrimination  Studies  with  ESTCP  with  Geonics  EM63  data  (Former  Camp  Sibert,  Alabama), 
with  static  MetalMapper  data  (San  Luis  Obispo,  California)  and  MPV  (Yuma,  Arizona). 

We  found  many  cases  with  close  targets  at  Camp  Beale  (Figure  26).  Approximately  40  flags 
had  less  than  0.5  m  separation  and  120  flags  less  than  1  m  -  this  does  not  include  cases  where 
multiple  anomalies  could  be  attributed  to  the  same  flag.  When  two  or  more  flags  were  found  in 
close  proximity,  all  targets  were  surveyed  with  the  same  beacon  reference.  All  the  data  were 
imported  together  and  the  data  were  masked  such  as  to  keep  only  the  data  associated  with  the 
target  of  interest  (Figure  21),  for  each  target.  In  the  case  of  overlapping  signatures  or  when 
multiple  anomalies  seemed  to  be  associated  with  a  flag,  we  tested  different  masks  as  well  as 
single  and  two-dipole  source  inversions  until  acceptable  fits  were  achieved. 


Figure  26:  Histogram  count  of  the  number  of  multiple  targets  as  a  function  of  the  target  separation 

(measured  with  the  beacon  positioning  system). 
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6.4  TRAINING  DATA 


The  classification  approach  was  based  on  the  capability  to  recognize  that  a  given  target  had 
similar  features  as  a  known  object.  The  method  therefore  required  a  library  of  known  target 
features  that  was  utilized  as  training  data  for  the  classifier.  The  training  data  consisted  of  target 
polarizability  features  gleamed  during  previous  MPV  surveys  and  site-specific  data. 

A  target  library  was  built  with  the  YPG  calibration  lanes,  which  offered  a  large  sample  set 
with  approximately  100  anomalies,  including  14  munitions-type  of  different  size  and  shape  and 
some  clutter.  The  munitions  had  been  buried  at  various  depths  and  orientations,  thus  providing 
the  opportunity  to  estimate  feature  vectors  variance.  This  library  was  used  for  the  initial  steps  of 
classification  although  its  polarizabilities  stopped  at  8  ms  as  opposed  to  25  ms  at  Beale.  The  data 
collected  at  Beale  in  the  test  pit  and  at  the  IVS  were  added  to  the  library.  The  corresponding 
unprocessed  data  were  made  available  to  all  analysts  for  them  to  develop  their  libraries. 

Site-specific  training  data  were  obtained  by  requesting  ground  truth  for  distinctive  targets. 
These  were  particularly  useful  to  characterize  unknown  UXO-like  objects  or  to  understand 
complex  target  signatures  and  thus  diminish  the  risk  of  misclassifying  unexpected  targets.  The 
requested  targets  would  appear  at  the  front  of  the  prioritized  dig  list;  therefore  it  was  in  the 
analyst’s  best  interest  to  wisely  choose  these  training  data  such  as  to  maximize  their  informative 
content  while  keeping  their  number  low. 


Figure  27:  Clustering  of  recovered  polarizability  decays  for  training  data  analysis. 

Top  left  panel  shows  feature  space  using  simplified  size-decay  parameters.  Training  data  from  calibration 
and  YPG  are  marked.  Two  polygons  define  regions  where  clustering  was  tested.  Right  panels  show  feature 
vectors  -  polarizability  decays  -  from  field  (grey)  and  training  data  (colors;  8-ms-polarizabilities  for  YPG). 
Bottom  left  panel  compares  a  field  target  (colors)  to  its  most  likely  known  target  (grey,  the  ISO). 
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A  clustering  algorithm  was  applied  to  identify  re-occurring  models  among  the  standard  one 
and  two  dipole  inversion  models.  The  method  was  based  on  comparing  all  recovered 
polarizabilities  with  one  another  and  group  self-similar  objects.  A  simplified  size-decay 
description  of  classification  model  space  is  shown  in  Figure  27.  Model  space  was  divided  in  two 
regions:  polygon  #1  only  enclosed  small  targets  and  polygon  #2  included  the  large  objects, 
where  UXO  are  more  likely.  The  clustering  algorithm  was  independently  applied  to  each  region. 
There  were  10  clusters  in  the  small  target  region  (Figure  28).  The  first  two  clusters  have  targets 
the  size  of  a  20  mm  projectile,  though  a  faster  decay.  The  third  one  has  spherical  shape  and  log- 
linear  decay.  The  fourth  cluster  corresponds  to  the  ISO  cylinder;  the  remaining  ones  are  smaller 
pieces  of  clutter.  In  the  large-targets  polygon  #2  (Figure  29)  the  seven  clusters  correspond  to 
items  the  size  of  the  ISO,  81  mm,  37  mm  and  60  mm  targets;  the  last  three  clusters  have  plate¬ 
like  characteristics  and  log-linear  decay  that  are  typical  of  magnetic  soil  models. 


Poly  1  -  Cluster  1  (30  pols) 


10° 


S*1 

1 

0.5  1  5  10 


Poly  1  -  Cluster  2  (1 1  pols) 


Poly  1  -  Cluster  3  (19  pols) 


Figure  28:  Clusters  in  the  small  target  region.  There  were  10  clusters  in  polygon  #1. 

The  first  10  panels  show  the  polarizabilities  for  each  cluster  and  the  mean  polarizabilities  (black  dashed  line, 
1  for  each  of  the  3  polarizabilities).  Titles  indicate  the  number  of  occurrences  for  each  cluster  in  brackets. 
The  first  5  clusters  include  many  targets.  We  required  at  least  4  occurrences  to  constitute  a  cluster. 

The  requested  training  targets  are  listed  in  Table  2.  Targets  were  chosen  out  of  the  clusters 
and  corresponded  to  end-member  models  with  the  largest  deviation  from  the  cluster  median.  The 
returned  ground  truth  indicated  the  presence  of  an  expected  TOI,  a  large  fuze,  which  had  three 
equal  polarizabilities  with  log-linear  decay  and  smaller  amplitude  than  the  ISO.  Classification 
was  devised  to  search  for  this  target,  which  later  was  deemed  to  be  harmless  (non  TOI). 


ESTCP  MR-201005:  MPV  Camp  Beale 
Demonstration  Report 


39 


March  2012 


Table  2:  Training  data  requests 


Target  number 

Cause 

Ground  truth 

45 

Poor  fit,  soil-like  decay  and  total 
polarizability  the  size  of  8 1  mm 

Frag  +  soil 

598 

Poor  fit,  soil-like  decay  and  total 
polarizability  the  size  of  8 1  mm 

No  contact  (soil) 

631 

UXO-like,  smaller  than  ISO 

Frag 

553 

UXO-like,  smaller  than  ISO 

Frag 

200 

Plate  like  scrap  or  soil 

Frag 

567 

Sphere  object 

Large  Fuze 

511 

Smaller  ISO 

Frag 

528 

Slightly  different  8 1  mm 

81  mm 

357 

Slow  ISO 

ISO 

556 

UXO-like,  20  mm  size 

Frag 

836 

Different  8 1  mm 

81  mm 

153 

Smaller  ISO 

Frag 
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The  classifier  was  trained  by  including  these  additional  data  to  prepare  the  first  prioritized 
dig  list.  Because  we  assumed  that  only  targets  within  30  cm  of  the  flag  would  be  validated  -  30- 
cm-radius  excavation  hole  -  we  excluded  targets  that  were  predicted  to  be  located  beyond  40  cm 
of  the  flag.  Distant  targets  were  labeled  as  “can’t  decide”  and  placed  after  the  stop-digging  point. 
Target  #4  was  one  such  target,  an  81  mm  projectile  that  was  predicted  to  be  located  45  cm  away 
from  the  flag.  The  target  ID  was  released  and  included  it  in  the  training  set. 

6.5  CLASSIFICATION 

The  classification  approach  was  chosen  after  examination  of  the  training  data  while 
considering  the  site-specific  magnetic  soil  disturbance.  The  following  guiding  principles  were 
adopted: 

-  Selection  of  models  in  classification:  We  assumed  that  classification  could  be  performed  in 
an  automatic  and  repeatable  manner.  Therefore  user  input  was  limited  to  verifying  that 
acceptable  fit  to  MPV  data  was  achieved.  Every  model  meeting  that  criterion  was  considered 
for  classification  without  further  user-dependent  pre-screening.  Given  the  effect  of  magnetic 
soil  on  the  MPV,  multi-target  inversion  tended  to  generate  soil  models  that,  depending  on  the 
predicted  depth  and  amplitude,  could  be  confused  with  105  mm  projectiles.  Decisions  were 
left  to  the  classifier; 

Selection  of  features:  By  analysis  of  the  training  data,  those  features  that  contribute  to 
separation  of  the  different  classes  (comprising  UXO  types  and  clutter)  were  selected.  This 
selection  involved  selecting  combinations  of  the  three  polarizability  decays  and  determining 
which  time  range  offered  the  largest  and  most  reliable  class  separation; 

Choice  of  classification  algorithm:  Through  analysis  of  the  training  data  the  best  performing 
classifier  was  selected.  We  focused  on  Support  Vector  Machine  (SVM),  which  historically 
had  yielded  the  best  results.  We  also  tried  a  rule-based  method  in  which  polarizability  misfit 
with  library  items  was  measured; 

Classification:  Anomalies  were  placed  in  a  prioritized  dig-list  by  using  the  classifier  to 
compute  probabilities  of  class  membership  for  unlabeled  feature  vectors.  The  probability  of 
membership  of  the  clutter  class  was  reported  on  the  dig  sheet; 

Number  of  UXO-classes :  One  or  more  UXO  classes  could  be  defined  when  seeking  UXO  of 
widely  different  sizes.  When  multiple  UXO  classes  need  to  be  used  (assume  there  are  M  of 
them)  then  one  can  either  train  an  (M+l)-class  classifier,  or  train  M  two-class  classifiers  and 
combine  the  results. 

Classification  was  generally  based  on  a  combination  of  a  size  and  a  time  decay  feature  for  the 
early  ESTCP  classification  studies  at  San  Luis  Obispo,  Camp  Sibert  and  Fort  McClellan.  More 
recently,  the  ESTCP  Camp  Butner  study  (ESTCP-201004)  and  the  MPV  demonstration  at  YPG 
showed  that  new-generation  sensors  such  as  the  Geometries  MetalMapper  and  the  MPV  could 
support  richer  feature  vectors  for  classification  owing  to  their  extended  time  range  and  improved 
recovery  of  all  three  polarizability-component  decays  (Figure  5).  Classification  was  improved  by 
using  a  more  complete  description  of  target  properties  as  opposed  to  the  simpler  size  and  decay 
pair.  The  main  polarizability  decay  (LI)  is  an  indicator  of  the  object  size  at  early  time  and  of 
material  properties  at  later  time  (wall  thickness  and  metal  types).  Secondary  (L2)  and  tertiary 
(L3)  polarizability  decays  contain  shape  information  along  the  target  secondary  and  tertiary  axes 

-  their  amplitude  should  be  equal  for  a  body  of  revolution  such  as  typical  ammunition.  One  of 
the  challenges  with  using  these  rich  features  was  to  determine  their  stability,  which  depends  on 
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data  quality.  LI  accounts  for  the  first  order  data.  It  is  therefore  the  most  stable  polarizability  and 
can  generally  be  used  over  its  full  time-decay  range.  L2  and  L3  describe  the  second  order  data, 
which  have  lower  amplitude,  especially  at  late  time.  Only  an  early-time  subset  of  L2  and  L3  was 
kept  for  the  classification  features. 

Polarizability  decay  features  can  be  compared  to  library  items  through  a  misfit  metric  or  a 
statistic  classifier  such  as  SVM,  which  can  accommodate  large  feature  vectors.  The  library  misfit 
was  not  used  as  the  primary  classifier  for  the  MPV  data  at  Camp  Beale.  Although  the  method 
yielded  remarkable  results  in  ESTCP  Pole  Mountain  survey  based  on  MetalMapper  data,  we 
found  the  method  to  be  less  effective  here  because  of  the  occurrence  of  magnetic  soil  models 
stemming  from  MPV  data.  Soil  models  usually  show  log-linear  decays;  their  relative 
polarizability  amplitudes  are  less  systematically  predictable,  although  it  is  expected  that  the  first 
and  second  polarizabilities  should  be  equal  and  greater  than  the  third  one  if  the  sensor  is  parallel 
to  an  even  ground  surface.  In  practice  there  were  numerous  cases  were  the  main  polarizability 
has  similar  amplitude  and  decay  to  81  mm  or  105  mm  projectile,  thus  resulting  in  low  misfit 
numbers.  Such  false  alarms  cases  could  not  be  avoided  using  a  simple  misfit  metric.  Therefore 
we  selected  a  non-linear  classifier  such  as  SVM  and  included  in  the  clutter  class  a  series  of 
automatically  selected  soil  models  identified  through  their  typical  log-linear  decay 
characteristics. 

Determining  the  number  of  classes  is  a  tradeoff:  when  using  few  classes  the  capability  to 
separate  targets  is  diminished  because  different  types  and  sizes  of  munitions  are  lumped  in  the 
same  class;  when  using  as  many  classes  as  there  are  munitions,  there  is  a  risk  of  being  too 
aggressive  if  misevaluating  in-class  variance  -  this  can  occur  if  signal-to-noise  ratio  (SNR)  is  too 
low  for  stable  recovery  of  target  polarizabilities,  and  if  variance  is  underestimated  because  few 
instances  of  each  munitions  type  are  available  for  training.  At  Beale  we  grouped  the  targets  of 
interest  in  four  different  classes  according  to  size.  We  defined  one  class  containing  only  105  mm, 
one  class  for  81  and  60  mm,  another  class  for  ISO  and  37  mm,  and  a  final  class  for  fuzes.  We 
used  the  SVM  algorithm  from  the  MatLab-enabled  Spyder  toolbox,  which  was  also  the  Sky 
Research  classifier  of  choice  for  the  Camp  Butner  and  YPG  studies. 

The  classification  approach  is  illustrated  in  Figure  30.  The  SVM  classifier  was  trained  on 
each  class  to  separate  that  class  against  the  other  classes  and  clutter.  The  four  SVM  classifiers 
were  then  applied  to  all  field  targets.  For  each  class  the  SVM  statistic  indicated  the  likelihood 
that  a  target  belonged  to  that  class.  The  SVM  statistic  was  greater  than  1  for  high-likelihood 
members  of  the  class  and  below  -1  for  the  least  likely.  Targets  could  be  given  priority  for  digging 
by  sorting  the  SVM  statistic  by  decreasing  order.  As  shown  in  Figure  30,  the  sorted  statistic  has  a 
L-shape  that  decays  rapidly  if  there  are  few,  distinct  members  of  a  class,  and  reaches  a  plateau 
for  unlikely  targets.  The  magenta  curve,  which  aggregates  all  UXO  into  a  single  class,  has  the 
slowest  decay  and  contains  more  targets  at  any  given  threshold  than  the  sum  of  the  other  four 
classes  at  that  same  threshold.  This  suggests  that  classifying  all  UXO  at  once  is  less  efficient 
than  using  multiple  classes.  We  used  the  SVM  classifier  in  multiple  stages,  starting  with  large 
targets  and  switching  to  the  next  smaller  class  below  a  given  threshold.  The  training  data  were 
analyzed  to  determine  the  appropriate  thresholds.  Similar  to  an  optimization  problem,  the  L- 
shaped  SVM  statistic  illustrates  a  trade-off  between  minimizing  the  uncertainty  that  there  remain 
targets  belonging  to  a  class  and  minimizing  the  number  of  targets  to  dig  out.  In  practice  the  SVM 
thresholds  were  automatically  set. 


ESTCP  MR-201005:  MPV  Camp  Beale 
Demonstration  Report 


42 


March  2012 


Figure  30:  SVM  classifier  applied  to  multiple  target  classes. 

For  each  class,  the  SVM  classifier  predicts  the  most  likely  class  members.  Each  curve  shows  the  sorted  SVM 
statistic  that  a  target  belongs  to  a  given  class  as  a  function  of  the  number  of  targets.  E.g.,  there  are  10  targets 
with  high  likelihood  of  being  105  mm  projectiles.  The  dashed  magenta  curve  lumps  all  UXO  in  the  same  class. 

This  process  was  repeated  for  different  feature  sets  to  reflect  reliability  in  parameter 
recovery.  The  first  iteration  included  all  3  polarizabilities,  then  the  first  (main)  and  second,  then 
the  first  and  third,  and  finally  the  sum  of  all  polarizability  decays.  As  showed  in  Figure  31,  the 
discriminating  capability  of  each  classifier  degrades  when  reducing  the  number  of 
polarizabilities.  After  each  SVM  stages  and  classes  were  exhausted,  the  remaining  targets  were 
sorted  as  a  function  of  their  misfit  to  known  library  items  to  produce  a  prioritized  dig  list. 

The  ranked  classification  anomaly  list  was  formatted  as  shown  in  Figure  32.  The  first  items 
on  each  anomaly  list  were  the  requested  training  targets.  Next  were  to  be  those  targets  for  which 
reliable  parameters  cannot  be  extracted  and  therefore  had  to  be  dug.  In  practice  data  quality  was 
such  that  we  considered  that  every  anomaly  could  be  analyzed;  non-contact  anomalies  were 
found  to  yield  soil  models.  The  list  continued  with  the  “high  confidence  munitions”  targets. 
Items  were  ranked  according  to  decreasing  confidence  that  the  item  was  hazardous.  Any  items 
that  were  analyzed  without  reaching  an  unambiguous  classification  decision  were  placed  next  on 
the  anomaly  list.  Finally,  all  items  that  were  confidently  classified  as  non-hazardous  were  ranked 
by  their  confidence. 
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Figure  31:  SVM  classifiers  applied  to  prioritizing  the  dig  list. 

Similar  to  the  previous  figure,  SVM  statistics  are  computed  for  each  class,  using  all 
polarizabilities  (L123),  subsets  (L12  and  L13)  or  the  sum  of  all  3  (Ltot). 


Rauk 

Anomaly 

ID 

Decision 

Statistic 

Category 

Dig  on 
First 
Pass 

Comment 

-9999 

2498 

-9999 

0 

1 

Can 4  extract  reliable  features 

2 

1 

2 

0 

Can’t  make  a  decision 

2 

0 

3 

0 

3 

0 

3 

0 

3 

0 

3 

0 

High  confidence  non-munition 

.03 

3 

0 

.03 

3 

0 

.02 

3 

0 

N 

.01 

3 

0 

Threshold 


Figure  32:  Format  of  prioritized  anomaly  list  to  be  submitted  to  ESTCP  Program  Office. 
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6.6  DATAPRODUCTS 


The  main  data  product  was  the  prioritized  dig  lists.  These  were  preceded  with  the  initial 
training  data  request  that  included  12  targets  (in  practice  the  ground  truth  seemed  to  provide 
indications  for  3  additional  neighboring  targets  that  were  included  in  the  training  set).  The  lists 
are  detailed  below: 

Stage  1  (slvl):  The  first  lists  had  originally  rejected  all  targets  further  than  0.4  m  away  (dig 
hole  was  set  to  0.3  m  radius).  As  a  result  we  missed  target  BE-4,  which  was  located  0.45  m 
away  from  its  flag.  Two  dig  lists  were  submitted:  one  automatic  rule-based  list  and  one 
edited  version  eliminating  obvious  non-UXO.  The  editing  stage  was  due  to  the  extremely 
conservative  choice  of  feature  vectors,  which  had  excluded  late  time  polarizabilities.  The 
multi-stage  classifier  used  SVM  with  LI 23,  LI 2,  then  LI  and  misfit  on  LI 23  afterwards. 

Stage  2,  version  1  (s2vl):  Dig  list  with  16  training  items  (12+3  and  1  from  slvl).  Based  on 
the  released  ground  truth  we  realized  that  greater  trust  could  be  given  to  late-time  data.  In 
particular,  the  LI  classification  stage  was  allowed  to  encompass  the  entire  time  range  (we 
had  originally  excluded  early  and  late  time  channels  to  avoid  potential  misclassification  in 
cases  of  early-time  signal  saturation  and  late-time  signal  weaker  than  soil).  We  changed 
multi-stage  classifier  to  LI 23,  LI 2,  LI 3,  LI,  misfit  LI.  We  found  that  many  obvious  non- 
TOI  picked  in  the  automated  slvl  list  spontaneously  disappeared.  The  final  dig  list  could 
have  been  more  efficient  with  a  less  conservative  first  stage. 

Stage  2,  version  2  (s2v2):  Dig  list  reduced  to  13  training  items  to  be  consistent  with  training 
data  request. 

Stage  3  (s3vl):  Final  dig  list,  same  as  s2v2  and  using  later  likely  thresholds  in  cascading 
algorithm  after  the  stop-digging  point. 
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7.0  PERFORMANCE  ASSESSMENT 

7.1  SURVEY  OVER  MAGNETICALLY  ACTIVE  SOIL 

This  section  examines  the  effect  of  magnetic  soil  on  MPV  data  for  classification  to  assess 
whether  the  adopted  survey  protocol  with  sensor  on  the  ground  was  the  most  appropriate. 

7.1.1  Effect  of  magnetic  soil  on  MPV 

The  intensity  of  the  magnetic  soil  effect  on  the  MPV  sensor  was  quantitatively  studied  with  a 
sensor  clearance  test.  After  identifying  a  seemingly  quiet  location,  we  acquired  background 
measurements  at  different  heights:  0,  4,  11,  15,  22  and  33  cm.  Cardboard  boxes  and  foam  pads 
were  used  to  stabilize  the  sensor  head  at  fixed  heights.  Recorded  time  decays  in  the  0.1-25  ms 
time  range  are  presented  in  Figure  33.  Colored  curves  show  the  soil  response.  Soil  affected 
receivers  exhibit  the  typical  log-linear  decay.  The  amplitude  of  the  soil  response  decreases  with 
sensor  height,  as  indicated  by  the  parallel  curves,  whereas  the  other  receivers  remain  in  the 
electric  noise  background  regime.  The  grey  and  black  curves  illustrate  the  recorded  signal  for  the 
central  soundings  that  were  collected  above  the  deepest  37  mm  and  81  mm  targets  for 
comparison  with  the  magnitude  of  the  soil  effect.  Target  response  is  always  larger  than  soil  when 
the  sensor  is  on  the  ground,  which  allows  target  detection.  However,  the  relative  difference  is 
less  than  twofold  on  some  receivers,  which  could  affect  inversion  results  if  the  background 
response  was  not  adequately  removed. 
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Figure  33:  Recorded  background  response  in  sensor  clearance  test  and  comparison  with  target  response. 
Colored  curves  show  the  recorded  background  response  for  different  heights.  The  black  and  grey  curves 
show,  for  comparison,  the  recorded  signal  for  the  central  sounding  above  the  deepest  81  mm  and  37  mm, 
respectively.  For  clearance  tests  the  sensor  was  placed  at  incremental  heights  above  ground  (4,  10,  15,  22  and 
33  cm).  Data  were  here  background-corrected  by  subtracting  the  in-air  instrument  response. 
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As  suggested  in  Figure  33,  soil  response  amplitude  depends  on  sensor  height  in  a  predictable 
manner.  We  found  that  this  dependence  could  be  empirically  approximated  for  this  test  with  a 
simple  linear  regression  between  the  logarithm  of  signal  amplitude  and  height.  Slope  factor  was 
fairly  consistent  with  an  approximate  value  of  -1 1.5  for  horizontal  components  and  -9  for  vertical 
component  receivers.  The  inferred  rule  was  used  to  assess  expected  background  response  at 
different  survey  height. 
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Figure  34:  Amplitude  of  soil  response  as  a  function  of  height  for  each  receiver. 
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7.1.2  Simulated  ground  clearance  effect 

The  effect  of  sensor  ground  clearance  was  examined  through  Monte  Carlo  simulations.  The 
effect  was  tested  on  UXO  and  clutter  of  various  sizes  to  assess  potential  degradation  in 
classification  ability.  Modeled  items  included  a  horseshoe,  a  small  fuze,  a  common  fuze  (of  the 
kind  detected  during  training),  and  37  mm  and  81  mm  caliber  ordnance.  The  items  were 
simulated  to  be  buried  at  random  orientation  and  depth  below  the  ground  surface  within  their 
expected  depth  range  (e.g.,  0.15-0.40  m  for  37  mm;  depth  for  fuzes  was  probably  exaggerated). 
Target  response  was  computed  using  the  polarizabilities  stored  in  the  library.  The  background 
response  for  each  sounding  (9  grid  soundings  and  one  background  sounding)  was  randomly 
drawn  from  a  set  of  200  validated  soil  background  responses  and  attenuated  for  the  appropriate 
sensor  clearance  (0,  0.05,  0.10  and  0.15  m)  according  to  the  rule  derived  in  the  previous  section. 
Hence  we  obtained  a  data  tile  for  each  target,  similar  to  the  field  data.  Background  response  was 
similarly  evaluated  and  compensated  by  comparing  the  weakest  target  sounding  and  the 
background  sounding,  computing  the  ratio  (bound  between  66%  and  150%),  multiplying  that 
factor  with  the  background  sounding  and  removing  its  value  for  each  target  sounding.  The 
process  was  applied  to  1200  Monte  Carlo  realizations. 
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Figure  35:  Comparison  of  predicted  and  prescribed  depth  in  sensor  clearance  simulations. 


Each  realization  was  inverted  like  field  data  using  standard  one  and  two  dipole  models. 
Ability  to  recover  depth  generally  is  a  reliable  proxy  for  assessing  inversion  quality.  The 
prescribed  (reference)  and  predicted  target  depth  are  shown  in  Figure  35  for  sensor  clearance  of 
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0,  0.05,  0.10  and  0.15  m.  The  best  results  are  achieved  when  markers  cluster  along  an  imaginary 
1:1  line.  The  upper  left  panel  shows  that  depth  was  best  recovered  for  small  targets  when  the 
sensor  was  placed  right  on  the  ground.  Deep  8 1  mm  targets  might  be  best  recovered  when  the 
sensor  is  raised  by  0.10  m  -  note  that  the  deepest  81  mm  at  Camp  Beale  was  buried  at  0.45  m 
below  the  surface. 

The  results  are  also  expressed  in  terms  of  misfit  between  the  prescribed  and  the  recovered 
polarizabilities  in  Figure  36.  The  best  results  correspond  to  the  lowest  misfits.  The  same  trends 
are  observed  with  the  two  metrics.  These  results  can  be  explained  with  the  fact  that  small  targets 
have  weak,  fast  decaying  responses;  therefore  SNR  matters  through  distance  to  sensor.  In 
contrast,  the  response  of  large,  deep  8 1  mm  targets  could  be  mixed  with  soil  because  of  apparent 
log-linear  decay  at  early  time;  therefore  it  might  be  best  to  raise  the  sensor  to  limit  ground 
disturbance.  In  conclusion,  given  the  targets  and  depths  encountered  at  Camp  Beale,  this  analysis 
suggests  that  there  was  no  significant  penalty  in  surveying  close  to  the  ground. 
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Figure  36:  Simulated  ability  to  discriminate  targets  measured  as  a  function  of  polarizability  misfit. 
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7.2  DETECTION  OF  MUNITIONS  OF  INTEREST 


Detection  requires  that  the  recorded  target  response  exceed  the  background  response.  Given 
that  the  MPV  survey  only  included  cued  interrogation  with  static  data,  detect-ability  was  here 
measured  by  the  presence  of  an  anomaly  in  the  cued  data  tile. 

All  targets  were  detected.  The  recorded  target  response  primarily  depends  on  the  target  size 
and  depth.  The  response  versus  size  relationship  is  shown  in  Figure  37,  where  object  length  is 
used  as  a  proxy  for  size.  The  amplitude  of  target  response  and  size  co-varied  and  were  modulated 
by  burial  depth.  There  were  no  large  targets  with  weak  responses  in  the  Camp  Beale  survey 
because  targets  were  buried  at  relatively  shallow  depths  compared  to  their  size.  For  instance  the 
deepest  81  mm  target  was  placed  at  0.45  m  depth.  All  TOI  had  SNR  well  above  100. 

The  portable  areas  at  Camp  Beale  had  36  so  called  “no  contact”  flags,  where  no  metallic 
debris  was  encountered.  These  generally  coincided  with  weak  instrument  response  lower  than 
SNR=100.  Reported  “no  contact”  with  larger  SNR  corresponded  to  nearby  metallic  debris  that 
was  detected  by  the  MPV  survey. 
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Figure  37:  Signal  to  Noise  Ratio  as  a  function  of  target  size. 

There  are  size  variations  in  the  TOI  class  -  for  instance  60  mm  mortars  are  13-24  cm  long 
(body,  or  body  with  nose  and  fins);  there  are  several  fuze  types  and  sizes. 
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7.3  CORRECT  CLASSIFICATION  OF  MUNITIONS 


The  primary  objective  of  the  demonstration  was  to  identify  the  presence  of  potential  TOI 
against  clutter.  Given  that  the  chosen  classification  approach  relied  on  comparison  with  reference 
target  signatures,  success  strongly  relied  on  recognizing  specific  target  types. 


7.3.1  Stability  of  the  recovered  target  parameters 


The  inferred  polarizabilities  for  each  TOI  types  at  Beale  are  shown  in  Figure  38.  The  tight 
distributions  of  polarizability  decay  curves  indicate  that  data  quality  was  sufficiently  high  to 
recover  at  least  two  of  three  polarizabilities.  Therefore  predicted  models  were  stable  and  could 
support  relatively  aggressive  classification  using  all  three  polarizability  decays  for  the  most  part, 
and  at  least  two  polarizabilities  for  all  targets. 
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Figure  38:  Recovered  polarizabilities  for  Camp  Beale  field  targets. 

For  each  field  TOI  except  the  fuze,  the  closest  model  to  the  IVS  reference  item  is  shown. 
Given  that  there  are  several  fuze  types,  the  standard  single  inversion  model  is  shown. 


There  was  some  variability  within  each  target  class.  For  the  60  mm  mortar,  the  main 
polarizability  decays  (red  curves)  suggest  two  distinct  target  types:  the  lower  amplitude  and 
faster  decay  correspond  to  the  60  mm  steel  body,  whereas  the  larger  amplitude  and  slower  decay 
curve  is  the  complete  target  with  aluminum  nose  and  tail  (Figure  39).  Retrieval  of  different 
decays  suggests  that  the  corresponding  dipole  models  represent  a  heterogeneous  target.  In  the 
past,  two-dipole  inversion  of  full  60  mm  mortars  had  generally  yielded  one  model  for  the  body 
and  one  model  for  the  tail.  This  might  have  not  occurred  here  because  the  presence  of  magnetic 
soil  might  have  had  a  greater  contribution  to  the  recorded  signal  that  the  isolated  aluminum  tail. 
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Figure  39:  Pictures  of  intact  and  partial  60  mm  mortars. 

There  was  some  variability  in  the  37  mm  class  too.  The  most  notable  outlier  was  target  #754, 
which,  as  shown  in  Figure  40A,  had  a  cracked  copper  band  relative  to  standard  37  mm  (panel  B) 
and  therefore  decayed  faster.  Targets  363  and  908  had  slightly  faster  late-time  decay,  which 
might  be  due  to  a  characteristic  pointy  nose  and  absence  of  ring  (Figure  40C-D). 
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Figure  40:  Picture  of  37  mm  outliers  with  faster  decay. 
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7.3.2  Prioritized  dig  list 

Performance  of  the  ranked  dig  lists  is  presented  in  Figure  41  in  the  form  of  ROC  curves.  In 
these  results  fuzes  were  first  considered  as  TOI  by  all  analysts  but  scored  as  non  TOI.  Hence 
there  remains  89  TOI.  The  top  two  curves  are  based  on  the  same  preprocessed  data  from  this 
study.  The  top  left  originates  from  the  classification  presented  here,  using  several  soil  strategies, 
12  training  labels  and  a  cascading  SVM  classifier.  All  TOI  are  recovered  after  approximately  95 
pieces  of  clutter  had  to  be  excavated  -  87%  clutter  rejection  rate.  These  results  would  be  even 
more  impressive  in  the  absence  of  fuzes.  We  reran  the  classifier  in  a  fully  automated  manner  to 
train  on  TOI  without  fuzes  and  found  that  only  48  pieces  of  clutter  would  need  to  be  excavated 
(Figure  42).  Returning  to  Figure  41  A,  we  chose  a  conservative  stop-digging  point  because  of  the 
presence  of  fuzes.  Note  that  the  “can’t  decide”  category  shown  in  yellow  corresponds  to  targets 
that  were  predicted  to  be  more  than  0.5  m  away  from  their  associated  flag. 
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Figure  41:  ROC  curves  for  four  different  analysts  with  fuzes  being  considered  to  be  clutter 
Panel  A  corresponds  to  this  study;  B  is  another  dig  list  by  SKY  based  on  standard  single  target  inversions 
(using  same  background-compensated  data);  C  comes  from  Dartmouth  magnetic  charge  models;  D  is  SAIC 
dipole-based  analysis.  Note:  all  analysts  were  looking  for  some  kinds  of  fuzes  when  preparing  their  dig  lists. 


ESTCP  MR-201005:  MPV  Camp  Beale 
Demonstration  Report 


53 


March  2012 


The  dig  list  illustrated  in  Figure  4 IB  corresponds  to  SKY  standard  processing  techniques, 
using  only  the  single-dipole  inversion  models,  the  same  training  items  and  polarizability 
matching  (misfit  to  library  items)  for  classification.  Performance  is  excellent:  all  TOI  are  found 
and  there  are  only  150  unnecessary  digs  -  80%  of  clutter  rejection;  the  stop  digging  point  was 
conservative  to  compensate  for  the  lack  of  soil  screening  and  multi-target  inversion.  These 
results  are  highly  encouraging  and  suggest  that  relative  simple  data  analysis  methods  can  lead  to 
solid  classification. 

The  next  two  dig  lists  were  created  by  researchers  at  Dartmouth  College  and  SAIC.  Both 
groups  chose  to  request  a  large  amount  of  training  data,  assumingly  to  compensate  for  their  lack 
of  familiarity  with  the  sensor  and  different  training  philosophy.  Both  ROC  curves  show 
remarkable  performance  with  near  perfect  dig  list  after  training.  Comparatively  more  clutter  was 
excavated  before  the  last  TOI  was  found  as  a  result  of  training.  However,  the  stop-digging 
decision  point  came  at  a  similar  rank  as  ours,  after  approximately  300  digs  (or  200  pieces  of 
clutter).  Together,  these  results  suggest  that  excellent  classification  performance  can  be  achieved 
with  MPV  data  and  that  hands-on  experience  with  the  sensor  is  not  necessary. 


Figure  42:  SKY  ROC  curve  without  fuzes  as  TOI:  Results  from  fully  automated  classifier. 

The  multi-stage,  cascading  clasisifer  is  illustrated  with  colored  segments  for  each  TOI  class.  The  89  TOI  are 
recovered  after  48  pieces  of  clutter  had  to  be  excavated  (95%  clutter  rejection  rate). 
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7.3.3  Ability  to  identify  small  fuzes 
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Figure  43:  ROC  curves  with  fuzes  considered  as  TOI. 


One  of  the  challenges  in  classification  at  Camp  Beale  was  the  ambiguity  on  fuzes,  which  lead 
to  unsatisfactory  ROC  such  as  the  one  shown  in  Figure  43.  Some  fuzes  were  originally  found  to 
contain  a  remaining  explosive  charge  and  labeled  as  potential  TOI,  while  others  did  not.  The 
difference  between  the  two  types  is  indistinguishable  with  an  EMI  system,  which  cannot  detect 
powder  content;  therefore  the  technology  is  not  suitable  for  separating  the  two  kinds.  Early 
ground  truth  suggested  that  some  of  these  fuzes  were  TOI,  which  lead  all  analysts  to  try  and  find 
them.  One  complicating  factor  was  the  occurrence  of  multiple  fuzes.  Most  fuzes  showed  a 
distinctive  spherical-shape  signature  with  three  equal  polarizabilities  and  log-linear  decay.  This 
re-occurring  target  features  were  noted  by  all  analysts.  We  selected  one  of  these  targets  for 
training  and  obtained  the  validation  shown  in  Figure  44  for  the  6  cm  fuze. 


Figure  44:  Fuze  TOI  selected  for  training. 

A:  Polarizability  decay  curves  of  fuze  (black  and  grey)  relative  to  all  known  TOI  type  (sorted  by  amplitude  at 
early  time:  105, 81,  72, 60, 37  mm  and  ISO  from  top  to  bottom). 

B:  Ground  truth.  Note:  this  target  was  first  labeled  as  munitions  debris  (MD)  and  reported  as  TOI. 
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Other  fuze  types  appeared  after  ground  truth  was  released  (Figure  45).  There  were  seven 
occurrences  of  a  5-cm  diameter  type  (A),  two  occurrences  of  a  4-cm  type  (B),  and  one  unique  3- 
cm  fuze  that  was  found  among  other  fragment  pieces  (C).  The  nature  of  our  classification 
approach  makes  the  latter  two  cases  virtually  impossible  to  find  because  our  method  relies  on 
finding  re-occurring  targets.  The  first  category  could  have  been  found,  had  we  been  looking  for 
such  small  targets.  Detailed  examination  of  our  Stage  2  dig  list  reveals  that  the  5-cm  fuze  in 
avertedly  appeared  on  that  list.  The  corresponding  polarizabilities  are  shown  in  Figure  46. 
Multiple  inversion  methods  raised  several  possible  models,  including  one  fitting  the  known  6-cm 
fuze,  and  one  fitting  an  unknown  type  that  turned  out  to  be  the  un-encountered  5-cm  fuze. 
Unfortunately  we  did  not  look  at  the  recorded  fuze  size  and  remained  unaware  of  this  fuze  type. 


Figure  45:  Pictures  of  small  fuze  types.  A:  5-cm  fuze;  B:  4-cm  fuze;  C:  3-cm  fuze. 
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Figure  46:  Recovered  polarizability  decays  for  one  fuze. 

Several  models  were  obtained  using  different  inversion  strategies.  One  model 
resembled  the  known  6-cm  fuze  type,  and  one  model  fit  the  5-cm  fuze  type. 


There  were  six  missing  fuzes  in  the  ROC  curve  shown  in  Figure  43.  Their  corresponding 
polarizability  decays  are  shown  in  Figure  47  and  compared  to  the  polarizability  decays  of  the  5- 
cm  fuze  type.  The  yellow-colored  panels  indicate  that  three  of  these  fuzes  would  have  been 
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recoverable  based  on  their  polarizabilities,  had  we  recognized  the  presence  of  the  5-cm  fuze  and 
trained  accordingly.  The  remaining  three  correspond  to  the  singular  4-cm  (#196  and  697)  and  3- 
cm  (#758)  fuzes  of  Figure  45.  These  could  not  be  recovered  through  our  classification  approach. 
Note  that  Dartmouth  researchers  did  find  all  fuzes  through  use  of  an  extensive  training  set  and 
close  scrutiny  of  all  polarizability  decay  curves  to  detect  any  target  with  symmetry  features. 


_Q 

TO 

hJ 

_ra 

O 

cl 


o 

cl 


o 

a 


10 


10 


10 


10 


10 


10 


1 


Target  ID:  482 


Target  ID:  196 


Target  ID:  805 


Target  ID:  758 


Target  ID:  697 


Target  ID:  558 


Small  fuze 

- Recovered  model 


Time  (ms) 


Time  (ms) 


Figure  47:  Polarizability  decays  for  the  6  missing  fuzes. 

Yellow-colored  panels  indicate  5-cm  fuzes  that  could  have  been  identified  through  adequate  training. 


7.4  ANOMALIES  THAT  NEED  RESURVEY 

Ideally  every  target  should  be  characterized  after  a  single  cued  interrogation  survey. 
Otherwise  it  might  be  cheaper  to  directly  excavate  that  target.  Field  procedures  were  such  that 
resurvey  should  never  be  necessary.  These  procedures  included  acquisition  of  a  3  x  3  grid  of 
cued  soundings  that  was  centered  on  the  target  flag  so  that  any  target  within  0.40  m  of  the  flag 
could  be  detected  with  sufficient  number  of  receivers  to  be  reliably  characterized.  In  addition,  the 
field  operator  was  asked  to  monitor  the  recorded  data,  if  possible,  to  infer  potential  target  offsets. 
In  practice,  over  1000  soundings  were  collected  every  day,  making  it  extremely  taxing  for  the 
operator  to  keep  constant  focus.  As  part  of  daily  Quality  Control  the  recorded  data  were  also 
gridded  to  ensure  adequate  spatial  coverage,  by  which  an  anomaly  should  appear  on  several 
soundings  and  the  anomaly  spatial  extend  should  be  delineated  with  data  reaching  the 
background  level.  Any  target  with  poor  coverage  was  marked  for  resurvey. 

The  nature  of  the  terrain  at  Camp  Beale,  with  side  slopes  and  tree  canopy,  sometimes  caused 
large  offsets  between  the  locations  of  survey  flag  and  those  of  the  picked  anomalies  from  the 
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EM-61  survey  map.  One  such  case  is  shown  in  Figure  18.  Although  that  target  could  be 
considered  as  No  Contact  for  the  absence  of  a  nearby  anomaly,  we  considered  that  the  anomaly 
located  in  the  SE  quadrant  could  be  further  characterized.  For  verification,  the  data  were  inverted 
and  suggested  a  large  deep  target.  After  resurvey  the  inverted  data  suggested  a  piece  of  clutter 
buried  closer  to  the  surface  at  a  distance  of  0.6  m  from  the  flag.  Frag  was  confirmed  by 
excavation.  The  crew  was  sent  to  recollect  10  flags  for  which  the  anomaly  was  far  from  the  flag 
or  seemed  to  be  undetected  (no  contact).  The  crew  was  asked  to  closely  monitor  the  recorded 
data  to  locate  the  origin  of  potential  anomalies  and  collect  data  to  cover  their  spatial  extent.  None 
of  these  anomalies  were  associated  with  TOI. 


7.5  LOCATION  AND  DEPTH  ACCURACY 

Outliers  in  predicted  depth  distribution  (Figure  48,  right)  were  due  to  deep  soil-like  models 
that  occurred  when  the  target  was  so  small  that  soil  was  more  likely,  or  when  the  data  supported 
the  presence  of  multiple  dipole  sources.  Soil  models  were  generally  predicted  at  greater  depth  to 
account  for  a  background  response  that  affected  all  the  data  collected  around  a  target. 
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Figure  48:  Comparison  of  the  predicted  and  observed  target  location  for  targets  of  interest  (TOI)  and  clutter. 
Left:  Difference  in  easting  and  northing.  Right:  Difference  in  depth. 


7.6  PRODUCTIVITY 

Ultimately  the  time  and  cost  associated  with  cued  interrogation  and  classification  should  be 
smaller  than  that  of  a  UXO-technician  with  a  shovel  -  safety  could  also  enter  that  equation. 
Survey  time  is  determined  by  the  number  of  cued  interrogation  soundings  and  any  additional 
time-consuming  field  procedure.  Classification  time  depends  on  the  amount  of  non-automated 
user  input  that  is  required  for  data  analysis. 


7.6.1  Number  of  soundings  per  target 

The  number  of  soundings  per  cued  interrogation  was  increased  from  6  to  10  as  a  conservative 
measure  to  ensure  adequate  coverage  through  a  simple  SOP  as  soon  as  flag-anomaly  offsets  were 
detected.  Fewer  soundings  are  necessary  for  unambiguous  target  characterization.  We  found  that 
as  few  as  three  high-fidelity  soundings  could  be  sufficient  with  the  data  collected  at  YPG. 


ESTCP  MR-201005:  MPV  Camp  Beale 
Demonstration  Report 


58 


March  2012 


We  re-inverted  the  Camp  Beale  in  retrospect  using  only  5  soundings  for  each  target.  The  5 
soundings  were  selected  by  drawing  every  second  point;  the  resulting  pattern  corresponds  to  the 
originally  planned  survey  protocol  with  one  center  shot  and  the  four  comers  of  a  square  and 
omitted  the  tilt  test.  Data  were  inverted  by  applying  standard  single  and  double-dipole  inversion 
methods  without  additional  soil  treatment.  Inversion  results  were  not  quality-controlled  and  were 
directly  sent  into  an  automated  classifier  (cascading  SVM  without  fuzes).  Results  are 
summarized  in  Figure  49  for  all  field  targets  and  detailed  in  Figure  50  for  TOI.  Recovered  target 
parameters  remain  stable.  Predicted  positions  stay  within  0.05  m  of  the  reference  10  sounding 
models.  Predicted  main  polarizability  amplitude  remains  within  50%  for  most  field  anomalies 
and  all  TOI.  Given  that  amplitude  logarithmically  scales  with  size,  these  variations  are  within  the 
in-class  variability  of  most  TOI  categories.  The  recovered  polarizabilities  remain  stable  for  most 
TOI  (Figure  50).  The  resulting  classification  performance,  illustrated  as  a  ROC  curve  (Figure 
51),  is  marginally  less  efficient  that  the  full  blown  approach.  There  was  no  QC  and  no  particular 
soil-related-noise  mitigation  strategy;  therefore  performance  could  likely  have  been  even  better 
with  optimal  models.  In  conclusion,  these  results  suggest  that  fewer  soundings  could  be  collected 
in  future  deployments  while  keeping  similar  classification  performance. 


Figure  49:  Recovered  target  parameters  with  5  and  10  soundings. 

A:  Difference  in  predicted  target  locations  between  5-10  soundings:  predicted  locations  generally  are  stable. 
B:  Percent  difference  in  amplitude  of  the  main  polarizability,  which  variations  (on  a  logarithmic  scale)  are 
indicative  of  target  size. 
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Figure  50:  Recovered  TOI  polarizabilities  with  five-sounding  surveys. 


Figure  51:  Comparison  of  ROC  curves  for  5  and  10  soundings  without  fuzes. 

The  thick  line  corresponds  to  the  automated  classifier  used  in  this  study,  without  the  fuze  class.  The  thin  line 
applies  the  exact  same  approach  based  on  5-sounding  inversion  models  without  QC. 
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7.6.2  Time  delays  in  the  field 

Most  of  the  time  in  the  field  is  spent  on  collecting  data.  On  average  it  takes  approximately  4 
minutes  per  target.  That  time  is  spent  between  moving  the  equipment,  positioning  the  beacon 
boom  and  acquiring  one  sounding  on  top  of  it  to  register  azimuth  (30  sec),  collecting  10 
soundings  with  6.3-sec  stacks,  waiting  2-10  sec  for  the  sensor  display  to  register  each  sounding, 
and  collecting  and  verifying  a  background  sounding.  There  are  several  ways  of  increasing 
productivity. 

Reducing  the  number  of  soundings  from  10  to  5  would  save  approximately  1  minute  per 
target,  thus  increasing  productivity  by  30%.  We  showed  in  the  previous  section  that 
classification  performance  would  not  be  significantly  affected.  Depending  on  environmental 
conditions,  in  particular  the  magnetic  soil  setting,  fewer  background  soundings  could  be 
collected  instead  of  taking  one  and  verifying  its  validity  for  every  single  target,  which  costs  15- 
20  sec. 

Productivity  could  be  further  increased  through  instrument  upgrades.  One  obvious  area  of 
efficiency  improvement  is  the  display  system.  The  MPV  DAQ  is  controlled  through  a  touch¬ 
screen  control  display  that  runs  Remote  Desktop  Protocol  (RDP)  over  a  local  Wi-Fi  network. 
Each  sounding  requires  the  operator  to  tap  on  the  touch-screen  display  and  wait  for  a  message 
indicating  that  stacking  is  complete.  The  system  suffers  from  time  lags  that  are  due  to  the 
capacity  of  the  touch-screen  display  to  register  the  tap,  the  limited  bandwidth  of  RDP  and  the 
stability  of  Wi-Fi.  For  instance,  we  found  that  a  Panasonic  Toughbook  was  better  at  registering 
taps  that  an  Apple  iPad.  We  anticipate  that  use  of  a  dedicated  touch-screen  field  monitor  using  a 
wired  connection  would  save  a  significant  amount  of  time  by  avoiding  these  three  layers  of 
complexity.  We  can  also  assume  that  a  dedicated  monitor  could  be  more  efficient  at  registering 
taps.  A  wired  monitor  would  provide  direct  connection  to  the  DAQ  without  communication  lags. 
This  could  save  at  least  30  sec  per  target  based  on  the  original  10  soundings.  An  audible 
notification  after  stacks  are  completed  would  further  speed  acquisition  by  telling  the  operator  to 
move  on. 

Beacon  azimuth  registration  is  time  consuming  because  the  MPV  head  has  to  be  lined  up 
with  the  beacon  boom  with  great  care.  At  least  20  sec  could  be  saved  and  positional  accuracy 
could  be  improved  by  installing  a  dedicated  compass  on  the  beacon  boom.  A  standard  digital 
compass  with  1 -degree  accuracy  would  be  more  reliable  than  a  careful  operator.  This  would 
require  minor  modifications  to  the  data  format  to  accommodate  this  new  data  stream. 


7.6.3  Data  analysis  time 

At  this  project  maturity  stage,  there  remained  many  data  processing  steps  for  which  manual 
user  intervention  were  needed  to  ensure  that  quality  and  stability  were  preserved. 

The  first  time  consuming  activity  was  the  manual  entry  of  recorded  file  numbers.  Each 
sounding  generated  a  separate  file.  File  numbers  were  consolidated  in  a  spreadsheet  that 
registered  the  file  numbers  for  the  beacon  azimuth  shot,  the  cued  survey  and  the  background 
sounding.  This  implied  over  1000  file  numbers  for  a  standard  90-target  daily  survey.  Although 
some  logic  was  implemented  to  automatically  populate  the  spreadsheet,  manual  entries  were 
necessary  when  multiple  targets  were  surveyed  with  the  same  beacon  station,  or  when  the 
operator  changed  the  number  of  cued  interrogation  soundings.  The  spreadsheet  was  subsequently 
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read  to  merge  the  data  files  and  create  a  data  tile  that  was  visually  checked.  This  process  could 
be  easier  if  a  strict  data  collection  was  enforced  so  that  file  number  sequence  would  be 
invariable. 

The  second  stage  was  computation  of  beacon-inferred  locations.  These  positions  were 
compared  to  the  GPS-predicted  ones  and  displayed  in  a  graphical  report.  Results  were  examined 
for  each  target  to  ensure  that  the  right  parameters  had  been  used.  The  process  is  now  stable  and 
should  require  less  manual  QC  in  the  future. 

Given  the  noted  effect  of  magnetic  soil,  time  was  invested  in  characterizing  that  effect.  We 
estimated  its  amplitude,  variability  and  spatial  distribution.  We  tested  different  data  processing 
strategies  firstly  to  remove  some  of  its  effect  on  the  data  that  would  be  inverted,  and  secondly  by 
testing  various  inversion  schemes  using  single  or  multiple  dipoles,  omitting  soil  sensitive 
receiver  data  and  deriving  model  constraints.  Specific  programming  codes  had  to  be  developed 
for  these  tasks. 

Occurrence  of  close  targets  also  caused  additional  work.  Close  targets  were  processed  as  part 
of  the  same  data  tile  to  allow  for  the  capability  to  jointly  invert  these  targets.  This  can  be 
necessary  if  their  signals  overlap.  There  were  also  cases  where  the  relative  location  of  field  flags 
and  detected  anomalies  caused  ambiguity  on  the  expected  target  to  be  characterized.  In  all  of 
these  cases  the  analyst  had  to  mask  the  data  to  be  inverted,  much  like  an  EM-61  survey,  and 
subsequently  examine  the  inversion  results  to  ensure  that  all  detected  anomalies  had  been  fit  with 
a  model.  This  process  was  particularly  time  consuming  when  manual  re -masking  or  changes  to 
model  constraints  had  to  be  done  and  data  had  to  be  re-inverted  and  QC-ed. 

Classification  was  relatively  straightforward,  given  that  a  multi-stage  cascading  method  had 
already  been  implemented  for  the  YPG  demonstration,  where  14  TOI  had  to  be  classified.  The 
one  complicating  factor  was  the  need  for  devising  means  of  identifying  and  weaning  off  soil 
models  in  the  automated  SVM  classification  framework. 

Overall,  we  expect  that  data  processing  would  be  faster  for  the  upcoming  demonstrations 
through  use  of  the  numerous  algorithms  and  QC  protocols  that  were  implemented  as  part  of  this 
survey. 
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8.0  COST  ASSESSMENT 

The  figures  presented  hereafter  reflect  the  costs  incurred  to  date  for  system  maintenance, 
software  modifications  and  upgrades,  shakedown  testing,  and  deployment  and  analysis  of  data 
collected  during  the  Camp  Beale  demonstration.  Deployment  costs  could  be  considered  to  be 
higher  than  expected  production-type  surveys  as  there  remained  some  development  tasks,  sensor 
data  characterization  studies  and  some  instrument  issues  that  had  to  be  addressed  in  the  field, 
thus  causing  delays  in  completing  the  project. 

8.1  COST  MODEL 

Time  and  resources  were  tracked  to  assess  the  cost  of  deploying  the  technology  for  future 
live  sites.  Note  that  some  of  the  cost  might  decrease  as  the  technology  matures  and  survey 
procedures  get  formalized.  In  particular,  most  of  the  data  QC  tools  and  data  processing  methods 
would  be  well  defined  and  not  require  additional  development.  The  site  had  particular 
environmental  conditions  (active  magnetic  soil)  that  required  in-depth  background  noise  analysis 
and  testing  of  alternative  parameter  extraction  and  classification  strategies  for  defeating  potential 
noise  disturbance.  Presence  of  close  target  was  also  a  new  feature  of  an  MPY  survey  and 
required  special  care  at  the  parameter  extraction  stage.  A  cost  model  is  proposed  in  Table  3 
based  on  the  survey  of  912  anomalies  and  15  days  in  the  field.  Retrospective  analysis  and 
reporting  are  not  included. 


Table  3:  Cost  model  for  MPV  demonstration  at  former  Camp  Beale. 


Cost  Element 

Data  Tracked  During  Demonstration 

Estimated  Costs 

Instrument  cost 

Component  costs  and  integration  costs 

•  Engineering  estimates  based  on  current  development 

•  Lifetime  estimate 

•  Consumables  and  repairs 

N/A 

N/A 

1.5  days 

Survey  cost 

400  hours 

Mobilization  and 
demobilization 

Cost  to  mobilize  to  and  demobilize  from  site 

•  Travel  to  town  base 

•  Daily  travel  to  site 

10  hours/ppl/stage 

2  hours 

Pre-survey  activities 

Before  mobilization:  Cost  of  one-time  event 

•  Sensor  verification  at  G&G  workshop 

On  site:  Cost  of  one-time  preparation  and  testing 

•  Programming  of  GPS 

•  Preparation  of  target  location  list 

•  Collection  of  training  data  (test  pit  with  3  ppl) 
Preparation  of  QC  tools:  Total  cost 

•  Hours  per  anomaly 

•  Personnel  required 

4  hours 

2  hours 

4  hours 

3x4  hours 

40  hours 

1  geophysicist 

Instrument  setup  costs 

Unit:  Cost  to  set  up  and  calibrate  (IVS) 

•  Hours  required 

•  Personnel  required 

•  Frequency  required 

0.33  hours/ppl 

2  ppl 

Daily 
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Data  collection  costs 

Unit:  Cost  of  collecting  &  recording  data  per  anomaly 

•  Hours  per  anomaly 

•  Personnel  required 

1/15  hours 

2  field  ppl 

Pre-processing  costs 

Unit:  Cost  per  anomaly 

•  Hours  per  anomaly 

•  Personnel  required 

1/15  hours 

1  geophysicist 

|  Discrimination  data  processing 

255  hours 

Data  extraction 

Unit:  Cost  per  anomaly 

•  Time  required 

•  Personnel  required 

1/20  hour 

1  geophysicist 

Parameter  extraction, 
including  inversion  and 
review 

Data  analysis  of  local  background  noise:  Total  cost 

•  Time  required 

•  Personnel  required 

Unit:  Cost  per  anomaly 

•  Time  required 

•  Personnel  required 

60  hours 

1  geophysicist 

1/20  hour 

1  geophysicist 

Classifier  training 

Define  strategy  for  defeating  soil:  Total  cost 

•  Time  required 

•  Personnel  required 

Unit:  Cost  per  anomaly 

•  Time  required 

•  Personnel  required 

40  hours 

1  geophysicist 

1/50  hour 

1  geophysicist 

Classification  and 
production  of  multiple 
prioritized  anomaly  list 

Unit:  Cost  per  anomaly  (includes  multiple  list  stages) 

•  Time  required 

•  Personnel  required 

1/20  hour 

1  geophysicist 

8.2  COST  DRIVERS 

The  MPV  was  developed  to  provide  a  moderate  cost,  reliable,  portable  sensor  with  advanced 
discrimination  capabilities  that  can  operate  at  sites  with  challenging  surveying  conditions.  As  a 
portable  system,  deployment  logistics  and  costs  for  transport  and  operation  are  quite  low  relative 
to  towed  arrays  or  other  vehicular-based  systems.  The  primary  costs  are  incurred  for  labor  and 
travel  for  the  operators,  and  the  primary  cost  driver  becomes  the  duration  of  deployment,  directly 
related  to  the  acreage  to  be  surveyed  as  well  as  the  difficulty  of  the  terrain  (steep,  rocky,  very 
uneven,  and  wooded  terrain  will  take  somewhat  longer  to  survey  than  just  because  it  is  more 
difficult  to  hike  across  these  areas). 

8.3  COST  BENEFITS 

The  primary  driver  for  developing  the  MPV  is  to  make  discrimination  feasible  at  a  wide 
range  of  sites  where  field  conditions  prohibit  the  use  of  cart-based  systems,  and  for  small-scale 
deployment  where  a  small  area  needs  to  be  surveyed  or  where  anomalies  need  to  be  resurveyed 
at  a  lower  cost  than  a  cart-based  system. 
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9.0  IMPLEMENTATION  ISSUES 


A  flow  chart  showing  the  managerial  hierarchy  and  the  relationship  between  the  principal 
investigator  (PI)  and  other  personnel  is  shown  in  Figure  52. 


Figure  52:  Project  management  hierarchy  with  Sky  Research  personnel  (blue)  and  other  personnel  (green). 

The  hierarchy  is  split  between  development  and  execution  components. 

The  primary  personnel  for  the  demonstration  were  Nicolas  Lhomme  (PI,  Sky  Research), 
David  George  (Electronic/Electrical  Engineer,  G&G  Sciences)  and  Benjamin  Barrowes 
(Scientist,  ERDC-CRREL).  N.  Lhomme  was  present  at  the  site  for  the  entire  project  duration 
(approx,  two  weeks).  D.  George  was  originally  planned  to  be  there  for  the  whole  time  too,  but 
had  to  leave  for  almost  a  week  in  the  middle  of  the  project.  B.  Barrowes  came  for  the  first  three 
days  to  help  with  setting  up  the  survey.  Jon  Jacobson  participated  in  the  last  10  days  of  field  tests 
as  a  field  geophysicist  and  main  MPV  operator.  Leonard  Pasion  and  Laurens  Beran  were 
consulted  on  data  inversion  and  classification  strategies;  Erik  Russell  was  the  project  manager 
responsible  for  cost  tracking. 
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APPENDICES 


Appendix  A:  Health  and  Safety  Plan  (HASP) 

Health  and  safety  procedures  will  be  followed  as  indicated  below,  and  will  also  comply  with 
the  ESTCP  guidance  for  this  demonstration. 

Applicable  local,  state,  and  federal  health  and  safety  laws  and  regulations 

Sky  Research,  Inc.  staff  on-site  at  the  Beale  test  site  will  comply  with  health  and  safety 
requirements  in  accordance  with  Code  of  Federal  Regulations  (CFR),  Part  29,  Section  1910.120 
and  any  site-specific  requirements  as  noted  during  site  orientation  or  other  direction  provided  by 
ESTCP  and  Department  of  fish  and  Game  representatives.  Although  29  CFR  1910.120  pertains 
to  personnel  conducting  activities  at  known  or  suspected  hazardous  waste  sites  and  may  not  be 
directly  applicable  to  the  planned  activities,  the  code  provides  a  reasonable  framework  safe  work 
practices. 

Potential  for  worker  exposure  to  hazardous  materials  and/or  other  hazards:  None  or 
minimal.  Any  site-specific  hazards  also  will  be  briefed  during  orientation. 

Physical  requirements  are  expected  of  workers:  Basic  fitness,  heat  resilience. 

-  Number  of  people  required  to  operate  the  technology:  Two 

Technology’s  history  of  breakdowns  or  accidents:  No  issues  to  date. 

Potential  effects  from  the  transporting  of  equipment,  samples,  wastes,  or  other 
materials  associated  with  the  technology:  All  components  of  the  technology  are  inert,  to 
the  exception  of  Li-ion  batteries  that  must  be  shipped  by  ground  according  to  federal 
regulation. 

Impact  of  technology  on  surrounding  environment:  None.  Technology  is  non  destructive 
and  man  portable. 

Closest  medical  facility 

Rideout  Memorial  Hospital 
726  4th  Street 
Marysville,  CA  95901 
(530)  749-4300 


Directions:  The  hospital  is  located  28  miles  or  45  minutes  away. 
Waldo  Rd 

1.  Head  north  on  Waldo  Rd  toward  Smartville  Rd 

2.  Slight  right  at  Smartville  Rd 

3.  Turn  right  to  stay  on  Smartville  Rd 

4.  Turn  left  at  CA-20  W 


1.7  mi 
4.6  mi 
1.0  mi 
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5.  Turn  left  at  CA-20  W/CA-70  S/B  St  (signs  for  Yuba  City/Sacramento) 


19.7  mi 


0.2  mi 


6.  Take  the  1st  right  onto  9th  St  (signs  for  CA-70/CA-20/Yuba 
City/Sacramento) 

7.  Turn  left  at  E  St 

8.  Turn  right  at  5th  St 

9.  Turn  left  at  H  St 


0.3  mi 
0.3  mi 
0.2  mi 
420  ft 


Figure  53:  Directions  to  Rideout  Memorial  Hospital  Facility 
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Appendix  B:  Points  of  Contact 


Points  of  contact  (POCs)  involved  in  the  demonstration  and  their  contact  information  are 
presented  in  Table  4. 


Table  4:  Points  of  Contact  for  the  MPV  Demonstration. 


POINT  OF 
CONTACT 
Name 

ORGANIZATION 

Name 

Address 

Phone 

Fax 

E-mail 

Role  in  Project 

Dr.  Nicolas 

L  horn  me 

Sky  Research  Inc. 

112  A,  2386  East 
Mall,  Vancouver,  BC 
V6T  1Z3,  Canada 

Tel:  541-552-5180 

Fax:  604-221-1055 

Nicolas .  1  h o m m c(a;sky  re search  .com 

pi 

Erik  Russell 

Sky  Research  Inc. 
445  Dead  Indian 
Memorial  Road, 
Ashland,  OR,  97520, 
USA 

Tel:  541-552-5197 

Fax:  541-488-5606 

Erik.  Russell(a!skyresearch.  com 

Project 

coordination 

David  George 

G&G  Sciences,  Inc. 

873  23  Rd 

Grand  Junction,  CO 
81505 

Tel:  (970)  263-9714 

Fax:  (970)  263-9714 
dgeorgetojggsciences.com 

Sub-contractor 

Dr.  Benjamin 
Barrowes 

72  Lyme  Road, 
Hanover,  NH 
03755-1290 

Tel:(603)646-4822 
Fax:(978)702-0448 
benjamin. e.barrowes(a;usace. army. mil 

Co-investigator 

Dr.  Herb  Nelson 

ESTCP  Program 
Office 

ESTCP  Office 

90 1  North  Stuart 
Street,  Suite  303 
Arlington,  VA 
22203-1821 

Tel:  703-696-8726 

Herbert. Nelsonikfosd.mil 

ESTCP  Munitions 
Management 
Program  Manager 

ESTCP  MR-201005:  MPV  Camp  Beale 
Demonstration  Report 


70 


March  2012 


